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SECTION Is INTRODUCTION 



The current national shortage poses serious problems to 
those responsible for selecting teachers in local school districts. 
On the one hand, there is always the danger that selection tech- 
niques may screen out applicants who would have proved to be com- 
petent teachers. False negatives, in the technical language of the 
personnel man. But, on the other hand, the very shortage serves to 
attract marginal prospects, and indeed, applicants who are serious- 
ly lacking in basic education themselves, let alone the ability to 
teach. 

It becomes essential, therefore, to examine procedures in 
the selection of teachers, to make certain that the techniques are 
reliable and valid, as well as understandable and acceptable to the 
community. 

In this context, the Board of Examiners of the public 
schools of the City of New York, the largest teacher selection 
agency in the country, has undertaken a series of steps to study 
the field of teacher selection for its relevance locally as well as 
nationally. The first step in this process was a survey of teacher 
selection policies and procedures in large public school systems in 
the United States. Aided by a grant from the United States Office 
of Education, this study has been completed and has been available 
in booklet form. 

The next step is the conference, which is summarized in 
these proceedings . A group of experts in the field of teacher se- 
lection was invited to meet for a two-day conference to explore and 
make specific recommendations with respect to needed research and 
action in teacher selection. This group of U5 met from November 16 
to November 18, 1966, at Grossinger‘s, New York. Again the United 
States Office of Education has made possible the conference and 
publication of the proceedings through a grant. 

The purposes of the conference were as follows: 

1. to suggest needed research in the area of teacher se- 
lection, 

2. to offer advice regarding designs and implementations 
of needed relevant studies, 

3. to review and discuss the results of the survey of poli- 
cies and procedures of selecting teachers in large 
public school systems in the United States, 



I*. to consider various aspects of examination procedures, 
such as the training of raters, types of rating forms 
to be used, independent vs. joint ratings, and setting 
of standards, 

» 

to consider the extent to which personnel selection 
practices in fields other than education can be applied 
to the selection of teachers, and 

6. to formulate desirable policies and procedures of se- 
lecting teachers, with some special consideration of 
teachers of children who are culturally different. 

A number of the invited experts prepared position papers 
prior to the conference. These are reproduced and the reader is 
invited to study them before reading the transcription of the con- 
ference proceedings. The latter present clearly the problems as 
they are faced daily by those who are struggling to staff schools 
in inner-cities - today's major educational problem. It is a dif- 
ficult, probably impossible, task to summarize fully the delibera- 
tions of this two-day conference. Hence, a careful reading of the 
proceedings may well bring the reward of new ideas or re-inforce- 

ments . 



Certainly one point emerges. Modification of selection 
procedures will not solve the problem of teacher shortage in inner- 
cities. The major- cities of the country, all represented at the 
conference, employ a variety of teacher selection procedures. 
Nevertheless, no city has solved the problem of staffing the 
schools in the inner-cities. The solution to this problem is part 
and parcel of the problem of large cities — housing, employment, 
transportation, discrimination, poverty, urban social pathology. 
Here is a major sub-problem which remains for intensive study. 
Given the facts of big city existence, how can we attract and se- 
lect teachers who are most likely to be successful with slum chil- 
dren? It is hoped that this problem will engage the attention of 
some of those who will follow the lead of this conference. 

A debt of gratitude is felt toward the participants in 
this conference. They came from great distances, indeed from 
Hawaii to Brooklyn and various points in between. Many prepared 
thoughtful position papers in advance. All worked hard for two 
days and their expressions are here recorded. Their ideas may 
serve to improve procedures here or there and to advance research 
in teacher selection. This would be the ultimate expression of 
gratitude for their service, making a contribution toward the 
education of our children. 

It is sad to note that Dr. Isidore Bogen, the Chairman 
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of the Board of Examiners and co-member of its Research Committee , 
passed away unexpectedly and in the vigorous prime of life* He was 
a responsible spark in the conceptualization of the overall plan 
for research in teacher selection of which the conference was a 
Vital part. He was looking forward with enthusiasm to the confer- 
ence, and indeed met with an accident as he prepared to leave for 
his conference, an accident which ultimately proved fatal. 

The conference proceedings are dedicated to his memory. 



SECTION II: POSITION PAPERS 



Selection Procedures at Standard Oil Company (New Jersey) 

Paul C. Baker 

Standard Oil Company of New Jersey 



Standard Oil Company (New Jersey) recognizes the crucial 
importance of well qualified employees at all levels, in all loca- 
tions, to its continued successful operation around the world. 

Great emphasis is given to establishing and maintaining sound, con- 
sistent practices to insure that the company will continue to at- 
tract and to select the best of possible candidates for all open- 
ings. This paper examines these selection procedures, their 
validity and their relevance to the selection of public school 
teachers . 



The comments that follow are not intended as a statement 
of official company policy nor are they intended to imply that 
these practices are rigidly followed in every detail in the hun- 
dreds of employment offices in all the countries of the world where 
Standard Oil has an interest. Rather they represent the collective 
aspiration of all our managers which gives direction to their con- 
tinuing efforts to improve their selection procedures. 

To better appreciate the foregoing qualifications perhaps 
a word about the Standard Oil Company (New Jersey) and six major 
regional affiliates. Humble Oil and Refining Company, Imperial of 
Canada, Esso Europe, Esso Inter-America, Esso Standard Eastern and 
Esso Africa. Each of these companies in turn is composed of affil- 
iate companies in each of the countries of the region. In addition 
to these are the major producing companies. Creole of Venezuela and 
Esso Libya* an international trading company, Esso International* 
Esso Chemicals, with subsidiary affiliates around the world* and a 
variety of other special purpose affiliates. In view of the size 
and complexity of this organization a very remarkable degree of uni- 
formity in selection procedures can be observed. 

To give emphasis and balance, to better relate it to over- 
all organizational needs, the selection process cannot be properly 
viewed in isolation, but must be seen in the perspective of a number 
of related functions: 

- Analysis of needs : How many people, with what 

qualifications, are needed today, next year, 
five years from now, twenty years from now? 

- Recruitment : Attracting the attention of pos- 

sible candidates to the challenges and oppor- 
tunities of employment with Esso. 
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- Selection’s Choosing from among applicants those 
most likely to succeed in the company. 

- Induction: The act of settling upon the condi- 

tions of employment and transforming an appli- 
cant into an employee. 

- Orientation s Acquainting the new employee with 
the organization, its goals , its structure , its 
functions, its relationships with other organi- 
zations . 

- Job Assignments Getting the new employee into 
productive work, setting goals and expectations. 

- Performance Evaluations Judging the degree to 
which the employee^ work performance satisfies 
the requirements of the job. 

The above outline is carried to this last step. Perform- 
ance Evaluation, because upon it hinges the validity of all the 
others, most particularly the validity of the selection procedures. 
It may also be relevant to the present discussion because of the 
differences that exist between the opportunities for sound evalua- 
tion programs in business and in schools . 

Each of the above functions merits detailed discussion 
to fully appreciate all the inter-relations, but only the selec- 
tion process will be dealt with here. 

There is a degree of selection present in the recruiting 
process. Standard Oil recruits from those colleges and universi- 
ties which offer training in those skills needed by our employees 
and, further, from only those that experience has shown to produce 
a larger proportion of successful employees. There is also selec- 
tion implied in the choice of media used to announce openings of 
various types. However, selection proper begins with the screen- 
ing of the application blank, or as labeled at Esse, the Qualifi- 
cation Record. At this step applicants are accepted for further 
consideration on the basic minimal job requirements such as age, 
education and training, and prior work experience. 

Having passed this first screening step applicants are 
invited to take a battery of qualifying tests. All test batteries 
have a demonstrated validity, validities ranging from .60 to * 75 .* 
They all span a wide spectrum of mental measurement a.nd have these 
elements in commons 

- A test of verbal intelligence. . . , 

■ ■« ■ * 



- A test of non-verbal or abstract intelligence. 



- A biographical information questionnaire repre- 
senting an extended ^objective interview on paper. 

- A test of job related judgment — i.e., sales 
judgment, supervisory judgment, etc. 

- A temperament or opinion questionnaire. 

- A test of job skills where appropriate — i.e., 
typing and dictation for secretaries. 



It is interesting to note the uniformity of test battery 
content and test validity whether considering a salesman in Norway, 
a mechanic in Libya, a manager in Peru, or a secretary in New York. 
Standard Oil's research over the past twenty years in yearly all 
noncommunist countries of the world seems to indicate that a 
broadly based, carefully researched battery of tests can become a 
most effective selection aid for any job. 

Applicants passing the test battery are then interviewed 
by three or more managers. In some locations the 
process is carefully controlled, the managers have had 
training and can be shown to contribute to the overall validity of 

the selection process. 



For a more detailed description of these selection pro- 
cedures a statement from th? Administrative Procedures Manual of 
an Esso Refinery is attached . 

The procedures outlined above are quite generally and 

successfully applied in all Esso selection progran®--there are 
obviously variations. But there are a number of ^P^jations, 
differences between business and education, that need tobeex 
plored to assess the applicability of these methods to teacher 

selection. 

In all of Esso's selection programs people are hired to 
fill a more -or- less specific entry job. The emphasis, however, is 
upon the selection of people with the capacity and the motivation 
to continue to grow and develop throughout their careers and to 
advance to successively higher level more responsible jobs. The 
implication is that a considerably broader range of qualification 
standards can be established and a considerably broader range of 
qualifications in people can be accepted than would be true 
people were hired only for one specific relatively unchanging jo . 

When a person is viewed as ineffective on one job there 
exists in most cases a wide variety of other possible jobs to 
which an individual can be assigned. What similar opportunities 
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exist for the ineffective teacher? 



To develop good selection procedures implies that there 
exists some standard by which the procedures nan be evaluated to 
determine if they are good. In business this implies measuring 
the quality of the people selected, and elabr -rate programs are de- 
veloped to provide this kind of data against-- which the selection 
program is compared. Although there is still much work to be done 
in the development of better more accurate performance evaluation 
programs, it still appears that very much more progress has been 
made in business than in education. What opportunities exist for 
such criterion research in the schools? 
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SELECTION PROCEDURES 



(From the Administrative Procedures Manual of a refinery.) 
Introduction 

To aid management in its function of recruiting, select- 
ing and placing the best possible qualified men in existing open- 
ings, the Employee Relations Division has established a set of pro- 
cedural guides which call upon the broadest possible resources of 
the staff and line organizations of the Company. 

The eight steps of the complete selection procedure and 
the departments responsible for them are: 

1. Announcement of openings - Employment. 

2. Application Screening - Employment. 

3. Testing - Training. 

1*. Medical Examination - Medical. 

5>. Interview Committee - Employee Relations and Line 
Management. 

6. Security Check - Plant Security. 

7. Former Employer and Credit Check - Employment. 

8. Summary Evaluation - Line Management and representa- 

tives of all the above. 

Associated with each of the above steps are detailed pro- 
cedures and a. set of qualification standards. The procedures are 
fixed; the standards vary, first, with respect to the specific open 
ings; second, with respect to the status of the applicant as a 
present employee, former employee or a new employee. 

For present employees selection standards may be lower 
than for new employees. 

The reasons for this are: 

1. For present employees such factors as work experience 
job success, motivation, flexibility, suitability as 
an employee are already a matter of record. 

2. It is the company’s desire to offer present employees 
continuing opportunities and challenges. 

3. When more applications are received than there are 
openings to be filled it is possible and reasonable 
to set higher standards for new applicants, standards 
which will reduce to a minimum the probability of 
failure of the new employee. 
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Following is a detailed description of each step in the 
selection procedure. The standards adopted for the 1959-1900 se- 
lection programs and the reasons for these standards are summarized 

in an appendix. 



Announcement of Openings 

After Management has determined a need for additional em- 
ployees for a particular operation or department, this need is ma e 
known through a series of announcements in the following order. 

The time between the several announcements may vary from a matter 
of hours to several weeks, and their order may vary with the spe- 
cific openings. 



1 To Management Personnel through normal channels of 
communications and the" Management Information Bul- 
letin. 



2. To the Union through discussions with appropriate 
members of Management . 

3. To Employees through the Daily Bulletin and the Esso 
News. 

To Former Employees who have been laid off within the 
past year. 



The objectives of these announcements are: 

1. To inform Management personnel and to enable them to 
answer questions from and inform employees and the 
general public. 

2. To inform all employees of the need and to advise 
them of the requirements for eligibility and the pro- 
cedure to follow to 'apply f°r transfer. 

3. To acquaint former employees with the situation inso 
far as it may apply to their recall rights or re- 
employment possibilities. 

The announcements of openings, in general, state as 
clearly and simply as possible: 

1. The nature of the openings. 

2 . Instructions on how to apply and where to get addi- 
tional information. 

3. Specific requirements which applicants will be 
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expected to meet such as: 

a. Have certain prior work experience. 

b. Have a certain educational level. 

c. Pass a medical examination. 

d. Pass a selection test battery. 

e. Be acceptable to interview committee. 

The statement of requirements will encourage only those 
potential applicants to apply who meet the requirements and will 
avoid unnecessary confusion and wasted effort by employment person- 
nel and by large numbers of obviously unqualified applicants . 

Application Screening 

Screening is the process of separating and eliminating 
from further current consideration applicants for employment or 
transfer who do not meet desired standards. It occurs during each 
of the eight steps in the employment procedure. This section deals 
with the preliminary screening of applications prior to that accom- 
plished at other steps in the selection procedure. 

Employment maintains an open door policy toward the pub- 
lic and will accept applications "from anybody at any time." Most 
applicants do not request an interview, but one will be arranged 
for anyone who desires it. The Employment Interviewer reviews the 
application, explains our procedure and current employment outlook 
and usually makes a note on the application of his over-all impres- 
sion of the interviewee. The application is retained for one year 
for consideration in the event openings should occur. Unless re- 
newed, it is then destroyed. This also applies to applications of 
present employees for transfer. 

Initial screening of employee-applicants for transfer or 
of former employee applicants is usually accomplished by examina- 
tion of seniority and other personnel records and by interview. 
These people sometimes voluntarily withdraw their application after 
an explanation and discussion of the openings and requirements. 

Screening of applicants for "new employment" to determine 
which applicants should be invited to take tests is accomplished 
largely by inspecting application forms. Desired standards are de- 
termined first. The criteria used in this preliminary screening, 
will vary with the type and number of openings and the availability 
of qualified applicants. They may include such factors as: 

1. Age. 

2. Education and training. 

3. Prior experience. 
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1*. Location of residence. 

5>. Relationship to employees or annuitants. 
6. Physical proportions (height and weight). 

) 

Testing 



The purpose of the testing program is to collect from the 
applicant by means of group administered paper-and -pencil tests 
such supplemental information regarding his mental abilities and 
attitudes as will be useful to management in deciding if the appli- 
cant is likely to succeed as an employee. 

The kinds of test measurements obtained and the questions 
they help to answer are: 

1. Intelligence - can the man learn to do the job? 

- will he be able to keep up with changes in the job? 

- can he be expected to progress to higher jobs? 

- will he show sound judgment in crucial situations? 

2. Educational Achievement - does the man have the basic 
education upon which to build? 

- does his achievement conform to his years of school- 
ing and his intelligence? 

- does his achievement show that he is highly moti- 
vated? 

3. Attitudes - does he have the attitudes, opinions, and 
personal history that have been shown to be character- 
istic of our successful employees? 

- does his judgment about himself and other people 
agree with the principles of human relations prac- 
ticed in the Company? 

ii. Job Knowledge - does the man know the principles and 
theory of his craft? 

- does he know the standards of practice of his craft? 

The tests comprising a particular battery and the passing 
scores on these tests are prescribed by management in terms of the 
needs of a specific selection program. 

Passing the test battery qualifies the applicant for con- 
sideration in succeeding steps in the selection procedure. 

To pass the selection test battery means to achieve a cer- 
tain minimum score on each test in the battery. Passing scores are 
determined by considering: 
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1. The content of entrance training programs. 

2. The norms of present and past incumbent groups. 

3. The job requirements, present and future. 

1*. The number of applicants. 

5>. The status of the applicant: present, former or new 

employee . 

The administration of the testing program is carried out 
by the Training Department. 

1. Testing periods are scheduled both on week days and 
on Saturdays. Applicants are notified by the Employ- 
ment Office of the optional testing periods open to 
them. They are asked to indicate in which period 
they will take the tests so that preparations can be 
made for them. 

2. The test battery is administered to small groups of 
from l£ to 2£ applicants in a room. Several such 
small groups may be tested simultaneously. Every 
precaution is taken to insure uniformity in test ad- 
ministration from group to group. 

3. The tests are scored immediately upon completion of 
the testing. Rosters of the test scores are prepared 
and distributed to the Head of the Employee Relations 
Division, the Head of the Employment Department and 
to members of the Interview Committees. Passing 
scores and individuals achieving these scores are 
indicated on the rosters. 

U. Applicants failing to pass the tests are notified by 
the Employment Department. Present employee appli- 
cants are notified through their Department Heads. 
Those passing the tests are notified of the time and 
place to report for a medical examination and an in- 
terview by the Interview Committee. Other than an 
indication of passing or failing the tests, the re- 
sults are not, in general, discussed further with the 
applicants. Present employee- applicants may discuss 
test results with their department head. 

S>. In the final Summary Evaluation step of the selection 
procedure the test results for each individual appli- 
cant are reviewed together with data collected in the 
other steps of the selection procedure. A member of 
the Training Department is present for more detailed 
analysis and interpretation of tests scores as they 
are related to other, data. 
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Medical Examination 
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The purpose of the medical examination is to determine 
and evaluate the general health status of the applicant as it is 
related to his ability to perform the job for which he is applying. 
Because employments are made on a career basis, factors that may 
affect future health are considered. 

1. Appointments for the medical examination are made by 
the Employment Department. As a convenience to new 
applicants and former employees, appointments are 
made for the same day as for the interview. 

2. For present employees applying for transfer, a medi- 
cal history is available. They are examined only if 
additional information is needed. 

Medical standards for a job are determined by considering 
the physical, emotional and psychological demands of the job. The 
degree to which an applicant meets these requirements is reported 
in term*? of: 

, 1. Over-all physical rating on five-point scale. 

2. Disabilities or physical limitations. 

3. The probable future state of applicant's general 
health . 

The data reported by the Medical Department are evaluated 
along with data obtained in other steps of the selection procedure 
in the Summary Evaluation step. A doctor is present at the final 
summary evaluation to advise those making the decisions. 

Interview 

Each applicant is interviewed by a three-man committee. 
The purpose of this interview is to gather data about the applicant 
that are available only in a face-to-face situation. The types of 
questions for which answers are sought are: 

1. How well does he express himself? . Listen? 

2. Does he show self-control? . 

3. Is he enthusiastic, highly motivated? 

h. How well does he think? 

S>. Does he have the ability to learn? . 

6. Will he be able to work well with others? Will he 
fit into the work situation for which he is being 
interviewed? 

7. Is he able to integrate his education and experience? 

8. Will his personal, appearance and manner add to or de- 
tract from his ability to work with others? 

13 
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Procedure 



Interviews are usually conducted by three-man committees 
on the basis that the evaluations made are subjective and 
three opinions are better than one. More judgments could be 
obtained by using larger committees, but having the applicant 
face such a large group might tend to upset him. Two members 
of the committee are experienced supervisors, department 
heads, usually, who are familiar with the work successful ap- 
plicants will do. The third member is usually selected from 
one of the Employee Relations Departments. This is because of 
Employee Relations* interest in personnel selection and, at 
the same time, provides something of an •» outside” evaluation. 

Thirty to US minutes are devoted to each interview. Each 
interviewer records his impressions of the applicant on a form 
especially designed for the purpose. In order to get inde- 
pendent judgments, interviewers record their evaluations with- 
out benefit of consultation with fellow interviewers. Each 
interviewer then ranks those applicants he has interviewed, 
again without consultation. Lists from the several inter- 
viewers are consolidated. Interviewers then discuss their 
evaluations among themselves and later with other members of 
management. (This is discussed in more detail elsewhere in 
this report.) 

Standards 



Applicants are compared to each other and are ranked. 
They are then (1) compared to the best people in jobs for 
which they are being interviewed, and (2) compared to the 
interviewer's own standards based on his experience as a 
supervisor. Based on these comparisons, interviewers deter- 
mine which people on the rank order list of applicants are 
acceptable, or meet the standards. 

Security and Gredit Check 

Experience indicates that the better employees are law- 
abiding and meet their financial obligations. In this regard, the 
best predictor of future conduct of an individual is probably the 
record of his past conduct. Therefore, it is important to consider 
past records in these fields. There are a number of private inves- 
tigating firms that provide this information for a fee. In the 
past, Dun and Bradstreet and Retail Credit have been used. More 
recently, however, State Police Department records have been 
screened and reports from the Baton Rouge Credit Bureau have been 
obtained . 



1U 



' State Police Headquarters maintains on IBM equipment rec- 

ords of all arrests, by name, in this and a number of other states, 
The Plant Security Department maintains a close relationship with 
State Police Headquarters. Names of applicants who have passed 
tests and are tentatively recommended by interviewing teams are 
taken by Plant Security Personnel to State Police Headquarters and 
screened for records of arrests. Detailed records are obtained on 
any applicant with an arrest record. 

Detailed written credit reports are procured from the 
Baton Rouge Credit Bureau. These reports itemize credit transac- 
tions the applicant has had in Baton Rouge and provide a good means 
of evaluating "how he handles financial obligations." 

Former Employer Check 

Former employers and close acquaintances usually have 
good ideas of how an applicant "gets along" with other employees 
and other people. Former employers are also usually familiar with 
the type of work the applicant is qualified to do and the degree to 
which he applies himself as well as his work habits and other 
traits. Experience has indicated, however, that formal or written 
inquiries to former employers or references have resulted in very 
little worthwhile information. 

A more effective system for obtaining desired information 
is to telephone former employers and inquire about the character 
and work habits of applicants. In general, former employers are 
very free to discuss the applicant on an informal and personal ba- 
sis . These informal appraisals have been very worthwhile in the 
screening and final selection of employees. 

Since many applicants prefer that their present employer 
not know of their interest in employment elsewhere, our policy is 
to not contact present employers. In cases where an applicant has 
had no former employment, efforts are made to contact neighbors, 
professors, or some other persons who may have personal knowledge 
concerning the applicant's character. 

Summary Evaluation 

The purpose of the summary evaluation is to bring to- 
gether all the people who have contacted applicants for the purpose 
of making an over-all evaluation of the data collected in each of 
the preceding steps. In attendance at this meeting are representa- 
tives from: 



1. The Employment Department. 

2. The Training Department. 

3. The Medical Department. 

u. The Departments where the openings exist. 

The applicants who were interviewed by the Interview Com- 
mittee are discussed one by one. At this point final eliminations 
are made on the basis of one or more of the following; 

1. A police record. 

2. Poor credit report. 

3. Poor recommendation by former employers or references. 

U. Recommendation of the Medical Department. 

£. Low evaluation by the Interview Committee. 

All of the data assembled for each applicant are examined 
and discussed. Additional interpretations of the data are con- 
tributed by staff members. Critical impressions and incidents re- 
corded by the Interview Committee are evaluated and related to com- 
mittee rankings. From these discussions comes a ranked list of 
names of applicants recommended for employment. This list, to- 
gether with a summary of all data, is sent to upper Management for 
review and approval before offers of employment are made. Should 
the number of fully qualified applicants exceed the number of open- 
ings, the best of those qualified are made offers. If there are 
fewer qualified applicants than openings, further efforts are made 
to secure more applications and the process is repeated. 
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Personnel Selection Practices in Industry 
Douglas W. Bray 

American Telephone and Telegraph Company 



Personnel selection practices in industry cover such a 
wide variety of methods that no ample justice could be done to all 
of them in any relatively short presentation. This paper, there- 
fore will be focused on three methods which are used widely in the 
Bell' System for selecting the men and women to manage the telephone 
business in the future. These three methods are those which make 
use of (1) biographical data, (2) attrition, and (3) the assessment 

center . 



The Bell System Scholarship Study 

Since the Bell System recruits up to 2 ,000 college gradu- 
ates per year, the business has long given attention to. methods of 
selecting the most promising of those available. Starting as long 
aeo as the 1920's, studies have been made of the relation of per- 
formance in college to progress in management. These studies have 
always shown a definite positive relationship. Recently, however, a 
most ambitious replication of these studies was made -a replication 
which substantiated the findings. of the earlier investigations. The 
details of this study may be of interest. 



The Bell System College Record file contained data cards 
covering the approximately 30,000 graduates of accredited colleges 
employed by the System at the time this study was started. For 
various reasons, the data cards covering the following groups oi 
men were eliminated from the study: those graduating from college 

after 19#>, those hired five or more years after graduation, eve- 
ning school graduates, and those on leaves of absence. These ex- 
clusions reduced the study group to about 17*000. 



An initial problem was to decide on a criterion of suc- 
cess in the business. The main criterion used in this study is the 
annual salary earned by a man as compared to that earned by. others 
with the same length of service in the Company. Before . making such 
comparisons corrections were introduced to adjust for differences 
between salary levels in different parts of the country and between 
different departments. 



After these adjustments were made the salary distribution 
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for each length of service was divided into thirds. It was then 
possible to say for each graduate in the study whether he fell into 
the top, middle, or bottom salary third of all the college gradu- 
ates in the System who had the same length of service as himself. 

An attempt was made to secure the verified rank in gradu- 
ating class from the registrars of the many colleges from which the 
men in the study had graduated. The needed information was secured 
on approximately 10,000 men. The men were then classified as to 
the third of the class in which they graduated with an additional 
classification of those who were in the top tenth of their graduat- 
ing class. A comparison was then made between these scholarship 
breakdowns and the salary thirds described above. The following 
table shows the results. 



Rank 


in College Graduating Class 
Related to Salary Progress 


as 






Top Salary Middle Salary 


Bottom Salary 




Third 


Third 


Third 




Top Tenth of Class 


$ 1 % 


32% 


11 % 


100% 


Top Thii"d of Class 




31* 


21 


100 


Middle Third of Class 


32 


36 


32 


100 


Bottom Third of Class 


26 


31* 


1*0 


100 



The table reveals a decided relationship between rank in 
graduating class and progress in the System. Fifty-one per cent of 
those who graduated in the top tenth of their class and h$ per cent 
of those in the top graduating third were in the top salary third j 
this compares to 2 6 per cent of those in the lowest third of their 
graduating classes. On the other hand, only 17 per cent of those 
in the top graduating tenth and 21 per cent of the top third were 
in the lowest salary grouping as compared to 1*0 per cent of those 
who had graduated in the lowest third of their classes. 

These results parallel quite closely the results of an 
early Bell System study of the same sort reported in 1928. At that 
time, 1*8 per cent of the men in the highest scholarship third were 
in the top salary third as compared to 22 per cent of the men from 
the lowest scholarship third. At the other extreme, 2£ per cent of 
the top scholarship men were in the lowest salary third as compared 
to 1*7 per cent of those ranking in the bottom third of their col- 
lege graduating class. 

Colleges, even accredited ones, vary tremendously in the 
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rigor of their requirements for admission and for satisfactory aca- 
demic work. This factor strongly suggests that the relationship 
between rank in college graduating class and salary progress might 
be even more pronounced if some account was taken of the quality of 
the institutions from which the men in the study graduated . Quali- 
tative rankings of colleges are, however, hard to come by. For 
these reasons, a special classification of colleges was made for 
the purposes of this study. The classification was based on pub- 
lished materials and off the record discussions with college deans 
and placement directors. As a result it was possible to classify 
colleges as "above average," "average," or "below average." The 
following table shows the relationship between rank in graduating 
class and salary progress for men from the colleges in each of 
these three broad groups. In order to reduce the complexity of the 
table, the separate breakdown of the top tenth in scholarship has 
been eliminated and only the percentage of men achieving the top 
salary third is entered into the table. 

Rank in College Graduating Class and College Quality 

As Related to Salary Progress 

(Per cent of men in each category achieving top salary third) 



Scholarship: 





Above Average 


Average 


Below Average 




Colleges 


Colleges 


Colleges 


Class 


$» 


h2% 


h0% 


of Class 


38 


30 


28 


of Class 


31 


25 


23 


will be 


observed that taking college 


quality into ac- 



count does maxe a aixxereu«jc. j.hc ““““ — 7 ’ 

above average school is in the top salary third 38 per cent of the 
time as compared to 28 per cent for average men from below average 
colleges. It is important to note, however, that scholarship is a 
more important factor than college quality. The top third men from 
below average schools, for example, have done better in the busi- 
ness than the lowest third men from the above average schools. 



One other feature of the table is worthy of mention, in 
respect to both the scholarship and college quality classifications. 
The performance of the highest group exceeds the average group much 
more substantially than the average group exceeds the lowest group. 
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The difference between the performance of top third men from above 
average schools and top third men from average schools is 13 per 
cent (55 per cent minus 1*2 per cent), but this latter group exceeds 
the top third men from below average schools by only 2 per cent (1*2 
per cent minus 1*0 per cent). Again, top third men from above 
average schools exceed middle third men from above average schools 
by 17 per cent (55 per cent minus 3b per cent) while the latter 
group exceeds bottom third men from above average schools by only 7 
per cent (39 per cent minus 31 per cent). 

A man's college grades are an evaluation of his most im- 
portant activity during college - his academic work. It is widely 
believed, however, that a man's success in extracurricular activi- 
ties may also be indicative of his future, particularly his success 
in business. The men in this study were, therefore, classified in- 
to three groups according to the extent of their achievement in 
extracurricular activities. The following table, therefore, adds 
one further breakdown to the data. The men are now classified by 
scholarship thirds, college quality, and into three groups depend- 
ing on whether their campus achievement was classified as "sub- 
stantial," " some," or "none." Again, only the percentage of men 
achieving the top salary third is entered into the table. 

Rank in Class, College Quality, Extracurricular 
Achievement and Salary Progress 

(Per cent of men in each category achieving top salary third) 



Top Third of Middle Third of Bottom Third of 
Class Colleges Class Colleges Class Colleges 



Above 




Below 


Above 




Below 


Above 




Below 


Aver- 


Aver- 


Aver- 


Aver- 


Aver- 


Aver- 


Aver- 


Aver- 


Aver- 


age 


age 


age 


age 


age 


age 


age 


_age_ 


_age_ 


Campus 


















Achievement: 


















Substantial 67 $ 


1* 1 % 


50# 


52£ 


3Q% 


3h% 


37% 


25£ 


26% 


Some 6l 


1*1* 


37 


1*2 


33 


21* 


32 


21* 


18 


None 1*7 


37 


37 


35 


28 


25 


31 


21* 


20 



ihe table demonstrates that extracurricular achievement 
is related to salary progress. In all nine scholarship and college 
quality groups (the columns of the table), those with substantial 
achievement have fared better than the other men. Extracurricular 



achievement is somewhat compensatory for lower rank in class. If 
only men from above average colleges are considered, for example, 
middle third men with substantial campus achievement have fared a 
little better than top third men with no achievement (£ 2 P er cent 
in the top salary third as compared to H7 per cent). Bottom third 
men, however, from above average colleges and with substantial 
camms achievement have made the top salary third only 37 per cent 
of the time. In other words, consideration of a man s campus 
achievements is helpful in predicting his probable success in the 
System, but it is not by any means as strong a predictor as his 

scholastic achievement. 

The findings of the study strongly support the conclusion 
that scholastic achievement is a substantial predictor of progress 
in management in the Bell System. There can be no _ question but 
that college recruiting efforts will succeed more in their objec- 
tive of bringing capable future managers into the business the more 
they emphasize rank in college graduating class as a criterion of 

employment . 

Some Cautions About Biographical Data 

Of late there has been a resurgence of interest among 
certain psychologists in the use of biographical data to Produce 
an index of suitability for particular jobs. Some of this in- 
terest and the studies on which it is based seem to the Present 
writer to contain two types of danger. One of these is thatthe 
biographical items may be prejudicial, such as the size of home 
livid in, the number of books in the house, father's occupation, 
etc It would seem that every effort must be made to see that bio- 
graphical data used either rigorously or informally be free of such 
imDlications . In our Scholarship Study, for example, al °“f 
there was some statistical relationship between college quality and 
performance in the business, our recruiting policy is to ignore the 
question of college quality. There seems little doubt that college 
quality emerges as a predictive factor because, due to selective 
college admission practices, the students who go to s° me institu- 
tions have more mental ability than the students who go 
We have, therefore, recommended to our recruiters that a general 
mental ability test be given to all collegegraduatesintheem- 
Dlovment process. The emphasis here is on the individual ratrier 

than"* upon^the particular college which he ha ? o ^falters 

clear that many students' choice of college is based on factors 

other than their mental ability and motivation. 

The other danger in some of the current trends in so- 
called biographical data blanks is that some of the items are not. 



in fact, biographical but are really personality and attitude ques- 
tions. Recent Congressional hearings have emphasized the point 
that whatever the predictive value of such questions may be, there 
is strong public resistance to their being part of the employment 
process. 

Attrition 



Although the use of objective measures like rank in col- 
lege graduating class and general mental ability test score defi- 
nitely do aid in the selection of' a more promising group of college 
recruits, they are like all other selection methods - fallible. In 
our Initial Management Development Program the recruit is told that 
his first year in the business will be a serious trial period. He 
is informed that he will be given a challenging assignment under 
the supervision of a specially trained supervisor who is already a 
member of middle management and that he will be regularly and rig- 
orously appraised as he carries out his work. The recruit is fur- 
ther told that if he does not appear under this scrutiny to have 
the high potential which he was thought to have at the time of em- 
ployment that he will be told this and asked to leave. -It is sig- 
nificant in this program that the supervisors to whom such recruits 
will report are especially selected and especially trained for this 
aspect of their job. Such an important evaluation is not left to 
just any supervisor and even those selected are worked with regu- 
larly to assist them in the proper observation and appraisal of 
their young charges. 

The Assessment Center 

In 1<?£6 we started a long range longitudinal study of 
young business managers which we called the Management Progress 
Study. As a major research tool in this study, which is now in its 
eleventh year, we developed a business assessment center. In the 
course of operating this center in the research study, it attracted 
the attention of several department heads who saw it as a possible 
method for improving the selection of first-level supervisors. 
Certain modifications in the method were made to make it more ap- 
propriate for the actual selection of people, and a center devoted 
to this purpose was opened in the Michigan Bell Telephone Company in 
19^8. This first application was successful and today most of the 
Companies in the Bell System have assessment centers to process both 
male and female employees who are candidates for management . 

The assessment center method involves (l) definition of 
the qualities relevant to performance in the job 5 (2) selection or 






construction of techniques for eliciting behavior representative of 
these qualities; (3) systematic observation of the resulting be- 
havior by trained observers intimately familiar with job demands, 
leading to (U) judgments of the candidate’s strength in each qual- 
ity and in overall promise. In our case, observation of the job 
and intensive discussions with those who supervised the first- 
level job led us to select some 20 qualities (or variables) of im- 
portance in first-level management. These included such character- 
istics as leadership ability, administrative skills, decisiveness, 
independence, etc. 

The major techniques at the assessment center are a busi- 
ness game, a leaderless group discussion, an individual fact- 
finding problem, an individual administrative task known as the In- 
Basket, an interview, and a few paper- and -pencil tests of knowledge 
and mental ability. Assessees are brought together in groups of 
twelve for a period of approximately two- and -one-half days. The 
assessment staff consists of seven or eight second-level super- 
visors, who have been first-level management themselves and who now 
supervise the first-level job, and a third-level director. They 
are given up to three weeks of training before beginning; assessment 
work. After writing detailed reports about their observation of 
the candidates in each of the techniques, the staff meets to dis- 
cuss each candidate in. turn for a period of one to two hours. Dur- 
ing this discussion, each candidate is rated on the 20 character- 
istics around which the assessment is focused and, finally, rated 
at to his or her degree of promise for a first-level management 
job. 

Follow-up studies of the assessment process have shown 
that the method does, in fact, select a higher proportion of 
better- than- average first-level supervisors and also a considerably 
higher percentage of those who have potential for still further ad- 
vancement. In addition, the method has proven to be highly ac- 
ceptable not only to management but also to those assessed. 

The management assessment centers described above have 
processed well over 20,000 men and women in the Bell System. All 
of these candidates have been, those who have already been employed 
for at least several years by the business. Recently, however, we 
have developed assessment methods for the selection of salesmen. 

In this case, the process takes less time - a day-and-a-half . This 
sales assessment center has been applied to candidates for employ- 
ment and it has been found possible to conduct assessment on a 
pre-employment basis. 
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Relevancy for Teacher Selection 



The possible relevancy of the first two methods discussed 
above - biographical data and attrition - to teacher selection is 
obvious and it would seem that no detailed discussion is called for. 
One major problem, however, in using biographical data would be the 
necessity for it to be validated against performance on the teach- 
ing job. I gather, although I am no expert in this area, that cri- 
terion measures are not easy to come by and the lack of them has, 
in fact, been one of the major roadblocks in the path of improving 
teaching selection. I would like to return to this problem later, 
after I have discussed possible applications of the assessment 
center . 



Two major components of successful teaching would seem to 
be knowledge of subject matter and the possession of those personal 
qualities essential to imparting that subject matter to students. 
Assessment methods could certainly be devised to measure such per- 
sonal qualities. (It goes without saying, that achievement tests 
could measure subject matter knowledge.) Many of the qualities 
which we assess in the management assessment centers are relevant 
to the job of teaching. Among them are range of knowledge and in- 
terests, organizing and planning, personal acceptance and impact, 
interpersonal sensitivity, oral communications skills, ability to 
influence others, behavior flexibility, and so forth. It is im- 
portant to note that the assessment techniques designed to reveal 
these qualities do not, for the most part, require job knowledge 
nor do they resemble the job. They, or techniques like them, could 
be applied not only to the job of supervisor but also to that of 
salesman, research director, army officer, or teacher, to name only 
a few. 

The evaluation of people by means of such a technique is 
far more defensible and acceptable than the methods depending upon 
biographical items or ordinary aptitude measurement. The reason 
for this is that the assessment technique is similar to that of an 
achievement test rather than an aptitude test. It inquires whether 
the candidate can perform the job-like tasks before him not whether 
he has the aptitude for such tasks. If one fails, it is possible 
to make some developmental effort and try again. In the case of 
dead biographical facts or aptitude test scores, the individual has 
no recourse. 

There may be impressive practical difficulties in apply- 
ing an assessment center to applicants for teaching positions at 
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the time of employment. Leaving expense aside, it would be impos- 
sible to staff up to assess all the candidates who would have to be 
assessed in the short period of the year in which recruiting takes 
place. It is for this reason that we do not use the assessment 
center in regular college employment. It might be possible, how- 
ever, for prospective teachers to be assessed toward the end of 
their teacher- training, possibly by the training institution, and 
the results made part of the placement information provided by such 
institutions. It would also be possible, and this seems more fea- 
sible, for the teacher to be assessed during a probationary period 
on the teaching job. The assessment results could then be con- 
sidered along with observations of the teacher in the classroom and 
other relevant information in making a decision as to whether to 
re-employ the teacher in the following school year. 

Now to return to the question of criteria against which 
to validate such selection devices as biographical data, tests, 
etc., I would like to suggest that the assessment center can serve 
also as a criterion. I understand that in spite of many decades 
of effort, no satisfactory criterion of teacher performance has 
been found. Although it certainly would be preferable to have a 
good measure of actual performance in the classroom, improvement of 
teacher selection methods should not wait forever upon the develop- 
ment of something which may never materialize. As an alternative, 
one might accept as a criterion the degree to which the teacher 
possessed those abilities which teacher trainers, supervisors of 
teachers,- and teachers themselves would agree were relevant to be- 
ing a good teacher. You may remember that in our case we decided 
upon twenty such abilities in the managerial field. Assessment 
center procedures would certainly reveal the extent to which 
teachers did have these teaching abilities and would also certainly 
make it possible for teachers to be sorted into four or five groups 
as to their overall effectiveness . This classification could then 
be used as a criterion against which to validate selection 
instruments . 
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Personnel Selection Practices in New York City Civil Service 

Bernard Berger 

Department of Personnel 
The City of New York 



Procedures Currently Utilized 

The selection methods used by the New York City Depart- 
ment of Personnel do not differ greatly from those used in other 
civil service jurisdictions. Compared to industry, civil service 
selection methods tend to be more elaborate in form while more cir- 
cumscribed in application for actual selection. The latter stems 
from legal requirements that civil service selection methods must 
be competitive in nature and reviewable by outside authority. 

These conditions force the use of a linear scale for overall evalu- 
ation of candidates rather than a multi-dimensional analysis. 

The requirement of " reviewability 11 necessitates the use 
of relatively self-explanatory rating scales with considerable 
"face validity 11 and the production of reviewable records such as 
written test papers, rating guides, tape recordings, and work sam- 
ples. Further, as the courts have taken it upon themselves to de- 
termine the acceptability and value of answers to questions and the 
level of performance before oral boards, we have been forced to 
shift our emphasis, to some extent, from the development of methods 
useful for selecting the best applicants to the development of 
methods which will not be challenged by the courts. However, with- 
in the narrow range allowed by the law and the courts, we use many 
selection methods. 

In each case described, our tests are developed by exam- 
iners in the central personnel agency working from detailed job 
descriptions in consultation with people from the employing agency 
and with subject matter specialists. 

1. Training and experience tests are used to reduce the 
number of unqualified applicants, to insure that those 
candidates who pass the other parts of the examination 
are able to perform the duties of the position and to 
predict comparable ability or competency on the basis 
of historical evidence. 

The requirements vary from "high school graduation" to 
a Ph.D. in a specified field plus X years of specified 
experience. In some cases there are fixed requirements 
which must be met. In others, there are many different 
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sets of equivalents. The score on the test may either 
be "qualified" or "not qualified" or a numerical rating. 

The rating scales used to obtain the numerical rating 
may include in addition to credit for degrees and years 
of experience, evaluation of such factors as quality 
performance in previous jobs and creativity as indica 
bv nature of publications. The candidate's statements, 
where these will affect his rating, are checked to the 
extent warranted. This may mean, in some cases, field 
investigations by teams of expert examiners. 

In few cases, training and experience are the only tests. 
This is more frequently the case for high level positions 
where it is necessary to recruit as widely as possible, 
and to complete the process as soon as possible, and 
where an evaluation can be made on the basis of training 
and previous employment. These unassembled examinations 
have the advantage of not requiring candidates to come to 
New York City or to subject themselves to a formal es 
and to permit the completion of the selection process 
within days after applications are received. 

While the use of an experience test for determining abil- 
ity is probably the oldest form of employment test, is the 
most useful form to determine minimum acceptable compe- 
tence in a restricted range of occupations primari y 
skilled crafts which require an apprenticeship, and has 
the other advantages noted, we have found it useful 
most cases only for making gross discriminations among 
candidates. We tend to use only the most objective fac- 
tors (school completion, etc.) and a fixed coarse scale 
by means of which superficially reliable ratings may be 
obtained. Once one raises the question of the equiva- 
lence of different courses of study in different schools 
and other questions of similar nature, it becomes impos- 
sible to develop a rating scale, let alone obtain 
reliable ratings. 

Problems relating to the use of experience to Predict 
future performance were recognized very early in the 
lection of public servants. During the middle ages, po- 
destas, roughly equivalent to city managers, raine an 
experienced in city administration were selected on the 
basis of their records. However, they were usually hired 
not as individuals but as leaders of a team of adminis- 
trators which went along with them. We still don t know 



how to measure an. individual's contribution to the suc- 
cess of a complex program. Unfortunately we are unable 
to make general use of the experience of the Italian city 
states. 

2. Performance tests vary from testing a plumber's ability 
to wipe a joint and a welder's ability to make an over- 
head weld which will resist a given number of pounds of 
pressure, to the group oral type of combat or the ability 
to interview and calm an upset parent. We have also ex- 
plored the use of the in-basket tests and other simula- 
tion techniques. 

Where the test situation accurately samples the future 
work situation and reliably measures the ability to per- 
form the sample work, as in the selection for skilled 
craftsmen, typists, and stenographers, we have been 
reasonably successful. 

We are not as sanguine about our other tests of this 
type - particularly in supervision and management. A 
major reason why we are floundering is that, as indicated 
previously, we neither really know how to measure mean- 
ingfully management performance nor do TT e have the people 
who could apply the measures we have developed. The 
people who could use the measures objectively can't be- 
cause they don't know the person's work performance well 
enough. Those who do, know the person too well to be 
objective. One advocate of the "in-basket test," frus- 
trated by the fact that the ratings of observers of 
managerial performance are neither reliable nor highly 
correlated with ratings on the "in-basket test," has de- 
cided that the true measure of managerial ability is 
performance on the test and not on the job. On this 
basis, all tests would be valid. But then, this is the 
assumption that researchers in the training field make 
when they measure change in behavior resulting from 
training by change in test performance. 

3 . We make greater use of paper and pencil tests than of any 
other selection method. In form, they include just about 
every test type used anywhere. In content they range 
from simple one hour tests of literacy to tests running 
over two or more days attempting to determine how well 
candidates would perform, as the written test for Chief 
of the Fire Department. 



Until July 1 of this year, almost all of the tests were 
released to the candidates. This made studies of diffi- 
culty, reliability and validity of little value. As the 
test questions were made public, "face -validity” was of 
paramount importance. The use of items not obviously job 
related although similar to those used in standardized 
tests for the occupation has on occasions resulted in at- 
tacks by employee groups, citizen groups, and members of 
the City Council. 

We also conduct medical and physical tests. A qualifying 
medical test is part of every open competitive examina- 
tion. Physical tests are used in selection for positions 
which require extraordinary physical effort. 

5>. A probation period, which varies from three to eighteen 
months depending on the job, is part of the selection 
process. Some agencies make effective use of this tool 
while others do not use it at all. 

Validity of Selection Procedures 

Before discussing validity of our selection procedures, it 
would be useful to describe our goals. In New York City as in other 
jurisdictions, civil service tests were introduced to take the se- 
lection of public employees out of the hands of the politicians. 
Without doubt, where used, our selection methods do this. Second, 
the selection methods must be honest. That is, no one should be 
able to buy a position on a civil service list. Except for a fewer 
than a half dozen minor instances over a period of over 80 years, we 
have been successful in accomplishing this. Third, the tests must 
convince the candidates and the community that they are fair. That 
is, that they don't favor one identifiable group over another in 
ways which are obviously unrelated to valid prediction of job per- 
formance. Although we have reservations about this, the community 
as a whole seems to be satisfied. And finally, the test should 
distinguish, in rank order, ability to perform the specific job 
while at the same time selecting on the basis of potential for ad- 
vancement. As there is no possibility that universally acceptable 
criteria for job performance and potential for advancement can be 
established and successfully used for most of our jobs, it is ob- 
vious that we cannot measure the validity of the selection methods 
for the achievement of this objective. 

Even if we change this goal merely to increase the proba- 
bility that "competent" employees will be selected, evaluation is 
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not possible. As not all candidates are appointed - either they 
fail the test or are not reached for appointment or refuse to take 
the job - it is not possible to compare the abilities of those who 
pass the test and those who do not. Despite these difficulties, we 
do carry on test evaluation activities. 

1. We try to get from operating agencies as much feedback 
as we can on the performance of the employees selected . 
Prior to the preparation of any test, we try to get in- 
formation about the performance of employees selected by 
the previous test. As might be expected, supervisors 
tend to be most satisfied with the skills of the crafts- 
men selected by performance tests. We recognize that 
even if agency evaluation of performance were valid, the 
level of this performance may be more a function of the 
nature of the labor market than the selection method . 

2. We have made and will make routinely, now that our exam- 
inations are not released, studies of item difficulty, 
item - sub-test discriminating value and all other sta- 
tistics which can be derived from test results. This 
has made it possible to compare the characteristics of 
our tests with those of standardized tests. We have, 
for example, on several occasions used the .nation-wide 
test for Beginning Social Workers prepared by the De- 
partment of Health, Education and Welfare and compared 
the statistics obtained with those resulting from the 
use of tests prepared by our staff. 

3. Some agencies have agreed to study test results. For * 
example, the Department of Welfare made a study of the 
Case Workers appointed over a two year period to uncover 
relationships among such variables as test score, job 
performance, race, college attended, and turnover. On 
the basis of these results, we were going to do away 
with the written test. For a number of reasons related 
not to test- job performance correlation but to the other 
goals described previously, we were unable to do this. 
However, we are continuing our studies. Some weeks ago 
the Department of Welfare appointed approximately ^00 
Case Workers on a temporary basis pending the rating of 
the examination and the certification of the list. The 
Department of Welfare has agreed to keep close watch on 
the performance of these employees to determine whether 
or not there are any relationships between passing or 
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failing the written test and the other factors listed. 

For the first time, to my knowledge, in a civil service 
setting we will be able to compare the performances of 
all who passed a test with that of all who failed. 

lx. While technical validation studies are not feasible or 
profitable over the entire range of jobs for which we 
select, a rational questioning of all our assumptions and 
procedures is. At this moment we are examining our clas- 
sification to determine, series by series, whether or not 
all classes of positions in a series are needed in the 
formal Civil Service sense of conducting competitive ex- 
aminations between successive classes. If we conclude 
that they are not, we may dispense with civil service 
selection between some. 

We are also reviewing our experience requirements. It 
may be that our requirements have less to do with job ex- 
perience and more to do with maintaining , neat monotonical- 
ly increasing progression between successive classes. 

3>. Factors unrelated to the test may influence the kind of 
person selected more than the test itself., For example, 
not all people who file for jobs take them when offered. 
The number who drop out of the market increases with the 
length of time it takes to make a solid job offer. How 
many are lost? Who are these people? How would they 
compare with those who are finally employed? 

Our Case Worker experiment has already told us that where 
less than 60 % accepted job offers four to six weeks after 
they take the test, 9$% accept on the day of the test. 

The nature of the job offered and work conditions obvi- 
ously play important roles in self-selection. We are 
studying our technical and professional jobs to remove 
elements which could be performed by persons with less 
training. We hope to reduce our needs for skills in 
short supply and to attract more competent people to more 
challenging jobs. 

The labor market is a crucial factor; in some areas, we 
are looking for more people than there are in the market. 
In addition to job restructuring we have instituted for- 
mal trainee programs. For our own staff we no longer 
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seek for people with experience or training in personnel 
administration. We recruit capable college graduates and 
train them to be personnel managers. 

Relevancy of Our Selection Practices to Selection of Teachers 

How relevant are these practices to teacher selection? 
While most of our practices are directly applicable in the recruit- 
ment and selection of teachers, it would not pay to discuss all of 
them as many are already in use. 

1. The time gap between recruitment and job offer must be re- 
duced to as close to zero as possible. In the school sit- 
uation this means offering college students teaching jobs 
while they are still in school. In addition, use should 
be made of the "walk in" technique which we and other 
civil service jurisdictions have used successfully. This 
technique should be coupled with a selection method which 
would permit immediate selection or rejection. 

2. The recruitment area should be broadened as far as possi- 

ble. Some ways in which this could be accomplished, in 
addition to increasing recruitment efforts, are: using 

unassembled tests for selection of specialists $ having 
other jurisdictions conduct tests $ and accepting teachers 
qualified by other jurisdictions, perhaps subject to a 
probationary period. 

3. Stop releasing tests. This will make possible the im- 
provement of selection instruments through evaluation of 
test results, the reduction in the cost of the selection 
program, and the holding of more frequent tests. 

U. Recruit college graduates without special training and 
education and provide the needed training after employ- 
ment. Programs such as Head Start may make it possible 
to provide training and experience during summer vaca- 
tions which might be given graduate credit. Employ 
those who are good teachers in this program without 
further examination. 

5. Review all elements of the selection process and elimi- 
nate or change those which are not demonstrably helpful. 
Those tests which are not reliable should be discontinued 
or sharply curtailed . 

While I have not studied the matter closely, it seems to 
me a priori that the chance factor in classroom tests is 
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too great for comfort. It might be better to use a one- 
term probationary period (perhaps as an assistant teacher) 
during which the new teacher would be under close class- 
room observation and supervision and would be helped to 
make the grade by an experienced teacher especially as- 
signed for this purpose or by a supervisor. 

6. Expend greater efforts on exploratory research. Attempts 
to evaluate and reach definitive answers at the present 
time is probably both wasteful and futile. It would be 
more valuable at this stage to spend some money to un- 
cover the right questions rather than answers to the 
wrong ones. 
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The Relevance to Teacher Selection of Civil Service 
Personnel Selection Practices in New York State 

ThomaS L. Bransford 

New York State Civil Service Department 



Introduction 

Consideration of personnel selection practices in Ameri- 
can civil service is fraught with the twin hazards of overcomplica- 
tion and overgeneralization. One convenient generalization is that 
appointment or promotion is the responsibility of an appointing of- 
ficer in an operating agency. His selection however is ordinarily 
limited to a small number of eligibles standing highest on an eval- 
uative listing prepared by a central personnel agency for existing 
and prospective vacancies in a whole class of positions. 

This separation of evaluative and selective processes de- 
rives from the constitutional doctrine of separation of powers of 
the legislative and executive branches of government. Civil Serv- 
ice laws at both State and Federal levels have been interpreted by 
the judiciary as a legislative restriction on executive powers. 
Provision of laws or regulations limiting the appointing authority 
to a single eligible have been held unconstitutional by both State 
and Federal courts. Consequently most American civil service sys- 
tems have adopted the "rule of three" whereby the civil service 
agency affords the appointing authority a choice of three eligibles 
ranked highest among those available for consideration for a par- 
ticular vacancy. Some jurisdictions have further widened the zone 
of consideration, in recognition that important factors may not 
have been assessed precisely or that there may be differences among 
positions in the same class or simply in deference to managerial 
responsibility. For some classes of positions the evaluative 
process itself may be shared with or delegated to operating agen- 
cies, subject to procedural standards prescribed by the civil serv- 
ice agency. In some jurisdictions and agencies, however, the exec- 
utive branch has imposed on itself a "rule of one," with each ap- 
pointment restricted to the highest ranking available eligible. 
Personnel shortages in many occupational categories have made most 
of these practices and distinctions largely academic. 

Historically and traditionally the competitive civil 
service procedures were designed to set up waiting lists of many 
candidates for a limited number of positions which were more at- 
tractive than demanding. The competition contemplated was that of 
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many candidates demonstrating their relative merit, fitness (and 
patience) for a few coveted appointments. 



In recent years the public service has been engaged in an 
entirely different competitive situation, that of competing with 
other employers for a fair share of professional specialty personnel 
in scarce supply and a fair share of high potential beginners and 
generalists from larger and less specialized pools of manpower. A 
change in tempo has also been required— leisurely and elaborate eval- 
uation procedures have been superseded by continuous recruitment and 
early offers of appointment. 

The personnel selection process properly starts with at- 
tracting the attention and interest of sufficient numbers of ap- 
parently qualified personnel. If potential candidates are unaware 
of or unimpressed with or averse to the conditions and opportunities 
of employment, efforts must be directed to improving and publicizing 
attractions and to reducing deterrents. Salaries must be adequate, 
working conditions and fringe benefits must be competitive, and pro- 
fessional staff must be supported by sub-professional and clerical 
assistance so that they may devote themselves to the professional 
aspects of their jobs. There must also be reasonable opportunities 
to grow and develop on and off the job. 



Unless and until the positions to be filled are able to 
compete for the interest of qualified personnel there is no point to 
elaborate evaluation and selection devices and procedures. 




The New York State Civil Service system is concerned with 
entrance and promotion examinations, competitive and non-competitive, 
in the service of the State and in the service of many ^cal urnta 
of government. The State service alone comprises roughly 100,000 
positions in 3000 position classes. In addition, local jurisdic- 
tions and public authorities use the State's personnel examining 
services in keeping perhaps 7!? *000 positions filled in severa 
hundred distinct classes . 

Providing personnel examining services in establishing 
eligible lists to fill vacancies in these positions currently re- 
quires approximately U000 examinations annually involving around 
1ft ,000 applicants . The number of applicants with which the State 
civil service examinations are concerned is in the same genera 
order of magnitude as in the New York City civil service but the 
number and variety of position classes covered and numbers of 
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examinations required annually are many times larger. Compared 
with the total Federal civil service system, however. New York 
State is concerned with only a fraction of the numbers of candi- 
dates, positions, position classes and competitions. The personnel 
operations of the Federal system are kept within manageable limits 
by geographic decentralization and delegation of responsibility for 
promotion and occupational specialty examinations to Federal de- 
partments and agencies. Compared with the separate offices of the 
Federal Civil Service- Commission, Central and Regional, the New 
York State Department of Civil Service faces a similar total volume 
and variety of personnel selection problems in original appoint- 
ments and a much heavier load in connection with promotion 
examinations . 

Much of the State*s examining program is concerned with 
levels and types of personnel whose recruitment and selection have 
little relevance to the recruitment and selection of teachers . 

Examining prospective personnel for positions requiring 
little formal training or for which the labor supply is ample in re' 
lation to the numbers needed presents problems and justifies pro- 
cedures presumably of little pertinence to the selection for posi- 
tions requiring considerable formal training and for which the 
available supply of applicants compared to needs is limited in 
quality and numbers. 

The State Civil Service Department has minimal responsi- 
bility for evaluating candidates for classroom teaching positions. 
Only in correctional, welfare, and mental hygiene institutions are 
there such positions under its selection program. Teaching posi- 
tions in public schools throughout the State are under the juris- 
diction of the Board of Regents and outside the civil service per- 
sonnel selection system. The State Civil Service system, however, 
does include educational experts and supervisors in departments of 
education. State and local. The system also recruits and examines 
for many classes of professional personnel using methods and pro- 
cedures possibly relevant to the selection of classroom teachers. 

Most State civil service recruiting and selection exami- 
nations begin with a public announcement setting forth a general 
description of the positions to be filled, any restrictive condi- 
tions of residence, age or sex, the prerequisite education and ex- 
perience. required, the application procedure, and an outline of the 
nature and coverage of the various parts of the examining process. 

Typically, there is a formal written test on a pre-‘ 
announced date at designated examining centers throughout the 
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State of New York and by special request and arrangement _ at centers 
outside the State. For higher level positions or those involvi g 
interpersonal skills the examination may include an oral teot. F 
a limited number of classes of positions there may be a competitiv 
evaluation of training and experience beyond that required for ad- 
mission to the examination. 

There may also be other parts of the examination. Promo- 
tion examinations always include added points for seniority at a 
preannounced rate of so many fractional poin s per 
in the jurisdiction. They may include points for quality of j 
performance during the past year. Veterans preference credits are 
granted for a single occasion for each qualified wartime veteran wh 
wa^ a N ewYork resident at entrance into military service M^ical 
and physical strength and agility tests may be required . Usually 
there is a verification of claimed education and experience and a 
check of police records and inquiry into records of physical dis- 
ability, imprisonment or hospitalization for mental or emotiona 

disorders . 

Some or all of the various parts or stages of the exam- 
ination may be administered as successive hurdles with only those 
Dassing an earlier stage admitted to the later or the various parts 
may be administered to all candidates allowing strengths on ^me 
parts to compensate for deficiencies in other parts. Some parts 
may be rated in three distinct scoring zones, clearly passing, 
clearly failing or conditionally passing. 

The various parts of the examination are eventually com- 
bined into an evaluative listing or ranking of all the candidates 
passing the entire examination and surviving each part of th. 
amination on which there is a required and preannounced passing 

mark. 

In some examinations there may be more than one ordering 
or ranking of candidates, usually related to special requirements 
or options for some of the positions to be filled .^Some candidates 
mav qualify only on one of these options or specialties whereas 
others may qualify for two or more. There may betwoorroore lev- 
els of positions to be filled with some candidates willing to be 
considered or meeting educational, experiential or test standards 
for only one of the levels. A formal reranking or selective oertr 
fication of eligibles for a class, a practice sometimes followed 
Federal exams, O.S. and Canadian, is not allowed in New York State 
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civil service examinations, although of course appointing officers 
who avail themselves of the choice of three or more eligibles with- 
in the consideration zone may in effect he reranking such eligibles 
in accordance with their conception of the needs of an individual 
position they are seeking to fill. 

The written tests designed and used in State civil serv- 
ice examinations embody an attempt to effect a reasonable compro- 
mise between generality and particularity. The major emphasis has 
been concentrated on planning, developing and refining a manageable 
number of reusable multi-purpose tests which in various combina- 
tions and with varying weights and standards may be incorporated in 
many different examinations. Hundreds of position classes have 
important elements in common. Some knowledges and abilities are 
common to a number of classes at particular levels. Others are 
common to several levels of particular occupational families. Many 
classes involve such general ianctions as supervision, administra- 
tion, research and statistics, interviewing, investigation, or re- 
port writing or such basic abilities as reading comprehension, 
verbal, quantitative and abstract reasoning, interpretation of 
tables and graphs, scale and instrument reading or spatial percep- 
tion. 



The State’s selection program for professional and admin- 
istrative trainees illustrates a number of features of possible 
application to the selection of teachers. The basic requirement is 
graduation from a recognized college. Subject matter knowledge is 
not directly tested but consideration for specialized positions is 
limited to candidates meeting relevant course requirements. A 
five-part differential aptitude battery is administered to all can- 
didates using different weights and standards for groups of posi- 
tions with different needs. The five parts of the aptitude test 
battery were derived from a battery developed for the Federal Civil 
Service where weights and standards for different professional 
classes were those which most sharply discriminated different lev- 
els of job performance among employees with war-service duration 
appointments. An oral test to appraise ability to express ideas 
and to deal effectively with others is administered on a pass-fail 
basis as a condition of appointment to positions of administrative 
trainee. Eligibles without postgraduate training or professional 
level experience are considered for trainee level appointments at a 
salary of $6300 annually which matures on completion of a year’s 
service and on-the-job training to permanent appointments at a 
salary level of $667^ under present pay scales. Eligibles with 
postgraduate training or professional experience or with superior 
academic records may be appointed directly to the $667^ level. 

\ 

* 
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The Validity of Selection Practices 

The validity of selection practices may be defined in 
many different ways and different definitions of 

a oonsiderable fariety of practices for the whole ga»t of position 
classes in the public service would seem to verge on absurdity. 

Even whefthe appraisal of validity is UMtjdto d. ter»ining th 
correlation between a set of scores on a battery of testsand 
of independent observations of the major aspects of actualjob per 
formance in a single class of positions, the practical difficulties 

are formidable. 

Broadly speaking, the validity of selection batteries for 
a single cCf positions is limited by the following considers- 

tions : 

The extent to which they make distinctions among appli- 
cants which are not relevant to differences in work 
performance. 

The extent to which they neglect knowledges, abilities 
and personal attributes which are relevant to differences 
in work performance. 

The extent to which unpredicted changes take place in the 
motivation and development of individuals between their 
evaluations as prospective and actual workers. Th 
longer the interval before appointment the more important 

this factor becomes. 

The extent to which unpredicted differences occur in the 
nature or conditions of work from time to time or place 

to place. 

Errors of measurement or lack of reliability in the 
tests . 

Lack of reliability or inter-rater differences in judging 
the quality of work performance. 

The selection programs of the public service are not im- 
mune to these limitations. In attempting to minimize some of the 
limitations we undoubtedly magnify others. 

When the reliability of the criterion (factor f) is in 
the neighborhood of .7 and that of the test (factor e) is .9, the 
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highest validity co-efficient that could be expected would be .63 
even if factors a, b, c, and d were ideal. When new test batteries 
are experimentally administered to present employees factors c and 
d are inoperative but correlations may be limited by restrictions in 
the range of present employees accomplished by the original selec- 
tion process and by accelerated turnover of both poor and superior 
employees . 



Multiple correlation studies between selection test bat- 
teries and overall criteria of job performance in the public serv- 
ice have occasionally yielded co-efficients of correlation as high 
as .6 but .U is more frequent. 

When we talk of validity of selection practices we pre- 
sumably are concerned with considerations beyond a co-efficient of 
correlation between a set of predictor scores and a set of job per- 
formance criteria. The real value of a selection program lies in 
its effectiveness in attracting and securing the appointment of 
sufficient numbers of candidates who can and will perform required 
services at quality standards. This kind of effectiveness may be 
achieved with tests of moderate validity and may fail of achieve- 
ment with evaluative instruments whose statistical validity is as 

\ 

high as fallible criteria will permit. 

In the present state of the labor market for teachers, 
basic efforts are required to improve the inducements and reduce 
the deterrents for entering the profession, to identify teacher 
potential and to enlarge and improve training opportunities. Until 
these long-term goals can be accomplished, improvements in public- 
ity and evaluative techniques cannot be expected to provide the 
quality and number of teachers needed to maintain, let alone expand 
and improve, our educational systems. 



Selection Methods in the Federal Merit System 

Raymond Jacobson 
U.S. Civil Service Commission 



Any discussion of selection methods in the Federal merit 
system, to be intelligible, must be made against a backdrop of 
understanding of the nature of the Federal service. 

The outstanding characteristics of the Federal service 
are probably its size and its diversity. The executive branch in- 
cludes about 2.7 million people working in literally thousands of 
different occupations. The programs administered by the various 
departments and agencies range very widely. 

Federal employees not only deliver the mail and collect 
the taxes, they also forecast the weather j control air traffic; 
repair ships, tanks and airplanes; investigate, prosecute and im- 
prison criminals; inspect and grade food; provide medical care; 
control plant pests and diseases; buy and sell all kinds of goods; 

etc. We even have a fairly substantial corps of teachers the 

primary interest of this conference. Most of the teachers in the 
Federal service are employed by the Bureau of Indian Affairs in the 
Department of Interior to provide education at both the elementary 
and secondary level to those Indians who are wards of the govern- 
ment on reservations throughout the country. 

Another characteristic of the Federal service is the wide 
dispersion of Federal employment geographically throughout the 5>0 
States and, to an increasing extent, overseas in foreign countries. 
This means that our system must provide felxibilities to permit 
adaptation of the basic system to the varying problems of different 
labor markets depending on the nature of the job to be filled and 
its location. Contrary to most popular opinion, only about 12 per 
cent of the Federal civil servants work in the Washington, D. C. 
Metropolitan Area. 

The great bulk of the 2.7 million jobs are filled through 
the competitive civil service system. Because of the diversity of 
occupations and their geographic dispersion, as described above, it 
has been necessary to develop a system which, while it follows a 
common set of principles and broad policies, permits a great varia- 
tion in the ways in which these policies and principles are applied 
in different places and in the filling of different jobs. 

The basic principles which the Federal merit system uses 
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in operating its competitive examining program are not unique to 
the Federal service. They are essentially the principles which 
underlie most merit systems in this country and in many countries. 
Therefore, there is no real need to elaborate on them to any ex- 
tent. Fundamentally, they consist of. the following; 

a. Publicity must be given so that a reasonable amount of 
information is made available to citizens about the 
existence of vacancies. 

b. Interested persons who have learned of the vacancy must 
have a reasonable opportunity to make known their avail- 
ability for consideration. 

c. Realistic, reasonable, and valid standards of competence 
and fitness must be applied impartially to all persons 
who make themselves available. 

d. The standards must contain no test which constitutes dis- 
crimination based on factors other than competence and 
fitness. This includes the absence of any political test 
or political clearance of applicants . 

e. Selection must be from among those determined on the 
basis of the standards to be most competent. 

f . Each applicant should be able to learn what consideration 
was given to his application. 

g. Each applicant should have an opportunity to request and 
receive an administrative review of the consideration 
given to his application. 

The principles outlined above are applied, as indicated, 
in all of our competitive examinations whether they be for laborer, 
resident youth worker in the Job Corps, shipfitter, clerk, teacher, 
economist, or nuclear scientist. A very wide variety of selection 
methods are used in our total examination system. They include; 

(1) written tests designed to measure skills or knowledge, or to 
predict aptitude or learning ability; (2) performance tests to 
measure skill (in such disparate fields as welding or typing)'; 

(3) evaluations of experience or education based on application 
forms, samples of work, reference inquiries, qualifications . inves- 
tigation, etc.; and (i|) oral examinations on a group or individual, 
basis. The specific method or combination of methods used in any 



one examination is typically determined in advance by the process 
of occupational analysis leading to the establishment of the clas- 
sification or position structure for one or more related occupations, 
and, at the same time, to the qualification standards to be used in 
filling the jobs. 

For the purposes of this discussion it will be most 
fruitful if we concentrate on the nature of the examination and 
selection methods used for filling entry level professional and 
administrative positions . 

The Federal government hires about 2^,000 people each 
year who are entering into the beginning levels of a professional 
career. Broadly speaking, we follow two quite different techniques 
in conducting civil service examinations for these professional 
entry level positions. The first technique is one which relies 
heavily on a written test as an initial screening device. It is 
exemplified best by the Federal Service Entrance Examination, de- 
scribed in some detail below. The second technique is one which 
relies exclusively on an evaluation of the education and experience 
record of candidates. Both techniques require extensive recruiting 
effort to attract into competition as many top quality candidates 
as possible. We have long ago learned that good selection tech- 
niques cannot breathe quality into a lackluster set of candidates. 
Therefore, we consider recruiting a minimum essential to any 
successful selection program. 

The Federal Service Entrance Examination 



This single examination accounts for about one-half of 
all hires into administrative and professional entry positions. It 
is our best example of the first technique mentioned above, i.e., 
major reliance on a written test as an initial screening device. 
This examination is used for the filling of entry level jobs in 
over 100 professional or administrative occupations. 

The kinds of jobs covered include personnel, budget, 
social sciences, administrative, supply, claims examining, writing 
and editing, etc. The examination process starts with the adminis- 
tration of a written test. This is a general abilities test de- 
signed to predict the potential of an individual to learn and 
develop on the job and to perform at a considerably higher level of 
responsibility after on-the-job training and development. Those 
who are successful on the written test, typically about iiO per cent 
of those who take it, are asked to submit an application indicating 
their background of education and experience. Special provision is 
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made on -his application for indicating the college work performed 
by course title and the grades earned in college for each ' 

Those applicants who make a sufficiently high score on the written 
test andwho in addition have an academic record showing a B aver- 
age of a landing in the upper 2* per cent of their class are eli 

at thp hieher of our two entry grades, GS-7 . Other eligibles 
^e placed on thef S-5 list. Applicants who have additional educa- 
tion beyond the bachelor's degree or additional experience als may 

be qualified at the GS-7 level. (Purely on ‘ test re - 

this fall we are starting to waive completely the written tes 
auirement for anyone who has a 3.5 grade average or is m the upper- 
10 per cent of his class, and who will be willing to " 

ment at the GS-5 level. Careful follow-up studieswiil bemadeon 
all appointees under this provision before making it a permanent 
feature of the system.) 

The result of this process is the creation of a lar ;6e 
list of eligibles (about £0,000 last year) which is then available 
for certification to different government d epar tment s ^ and 
as they decide to fill vacancies. Such an undiver sified list would 
be of little use in matching individuals with specific jor q 
ments- therefore, a careful coding of the applicant s backgroun y 
subiect matter area, based either on the nature of courses taken or 
the experience, is performed as part of the rating process. This 
information along with test part scores is recorded “punch cards. 

When an agency request for eligibles is re0 ® i y® » . th ^ 

quirements identified by the agency are used to sort out those 
-i p-ibles whose background or test scores most nearly ma 
ne^s o? the specific job to be filled. Thus, this rather massive 
list of eligibles is used in a highly refined fashion in oj-der to 
relate closely the job requirements and the eligibles on the list. 

This program has been highly successful in terms of 
volume of competition we have attracted throu^ intensive recit- 
ing by the Commission and by the agencies. Generally, however, it 
has proved to be feasible only in those situations where there is 
not an extreme shortage of qualified people “the a or ^ 
where the test battery used is reasonably relatedto the career re 
rmirernents. There are many occupations in which the entry 1 
jobs are filled in quite a different way; among these are teachers. 

O ther Entry Level Professional Examinations 

Generally speaking, the entry level professional jobs 
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which we fill by examinations, other than the Federal Service 
Entrance Examination, are characterized by shortage from the stand- 
point of the Federal service and most employers today. The primary 
areas of short supply in the Federal Government are such fields as 
engineering, the physical sciences, mathematics, accounting, etc. 

The examination system for these entry level professional positions 
in short supply has been one which has excluded the use of the 
written test as part of the evaluation process. 

In the case of this kind of examination, it is quite evi- 
dent that our real problem lies not so much in measuring the rela- 
tive quality of those who apply but rather in getting enough qual- 
ified people for consideration. Therefore, the essence of the 
examining and selection function lies not in the derailed appraisal 
of individual backgrounds and the measurement of relative skill or 
knowledge levels but rather in the performance of a quality and 
effective recruiting job. The process followed therefore, for . 
example in the case of engineers, is one by which the examination 
announcement specifies the minimum requirements of the job in terms 
of education or experience. The recruiter, who is usually an 
agency official rather than a representative of the Civil Service 
Commission, has been designated as a panel member of a board of 
examiners and is thus authorized to evaluate candidates as welx as 
recruit. This recruiter then tries to do as imaginative and crea- 
tive a job as possible in interesting potential applicants in con- 
sidering employment with his particular agency. It has long been, 
evident that in these areas recruiting is most effective when it. is 
tied to a particular program. In fact, the most effective recruit- 
ing in many technical fields consists of describing to a class of 
seniors a fascinating technical problem which the agency has re- 
cently solved and not even mention the need of the agency for appli’ 
cants. This kind of recruiting, of course, can be done most ef- 
fectively by technical people experienced in the subject matter 
field for which they are recruiting. Frequently, teams are. used in 
which the technical recruiter is joined by a personnel specialist 
who is expert in recruiting technique rather than in the subject 
matter field. 

In this type of system in which the only basis we have 
for relative evaluation of candidates is their academic record or 
possibly some experience, we still face the problem of having to 
rank eligibles in their relative order of ability to do the work. 

In some cases, of course, this problem is not faced since it is not 
possible to have sufficient candidates to fill the jobs available; 
therefore, regardless of relative quality, all may be offered an 
appointment. If it is necessary, however, to establish a ranked 
list of eligibles we have devised rating schedules which provide 









for the uniform crediting of points to different types of back- 
grounds. Thus, for example, the graduate of an accredited en- 
gineering college will be given a higher score than the graduate of 
a non-accredited college who still meets the minimum standard. 
Likewise, the graduate with a high grade point average in his aca- 
demic work or with other indication of academic excellence based. on 
faculty recommendation or honors, etc., will be given extra credit 
and brought nearer the top of the list. This. type of rating 
schedule serves an effective purpose in that it provides for a 
ranking that relates closely to those qualities which have been 
proven to be predictors of job success. 

The key factors in a successful recruiting and selection 
program for entry level jobs in shortage category occupations. are 
(1) interesting and challenging job opportunities, and (2). ability 
to make a firm job offer promptly. The Federal civil service. makes 
this second factor possible by the technique outlined above $ i.e., 
the agency recruiter uciug &. j-j-cexiseu examiner can assign a rating 

and offer a job on the spot if he so wishes. If the shortage 

situation is temporarily or partially relieved, the recruiter's, 
authority to make job offers may be limited to only. those recruits 
who achieve above a particular score on the evaluation of education 
or experience . However , even this kind of mild limitation is 
rarely necessary in these areas. 

Interviews 

The selection method which I have talked least about in 
this paper is the one which is commonly considered to be the oldest 

and most widely used method of all the interview. While we 

still use the formal oral examination as part of the examination 
process in a very few situations in the Federal civil service sys- 
tem, we have been rapidly tending away from its use in this fashion. 

This is not to deny the importance of personal qualities 
or attributes in the prediction of job success in many kinds of oc- 
cupations. We believe that we can determine the degree. to which an 
individual possesses these necessary qualities better, in most 
cases, by a carefully designed inquiry directed to former super- 
visors, colleagues, etc. than by an interview. Therefore, we have 
tended to put more of our resources into the better design of in- 
quiry letters or into personal investigations than into improving 
interview techniques . 

As a safety valve in the system, we do permit agency ap- 
pointing officers to submit objections to an eligible based on the 
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results of an interview performed as part of the appointing offi- 
cer’s right to select one of the top three names certified to him. 
If the objections are based on a carefully structured interview 
with adequate documentation that tends to show the candidate to be 
significantly lacking in necessary personal qualities appropriately 
measurable by the interview, we will support the objection. This 

practice has saved considerable time, effort, and paper work 

with no indication of abuse so far. 

Conclusions 



The above describes two quite different selection methods 
in wide usage in the Federal service for filling entry level pro- 
fessional and administrative jobs requiring college level back- 
ground or equivalent. Both of these methods have been tested 
heavily in the intensive competition for quality personnel during 
the last decade. On the whole, while we are constantly changing 
and improving these techniques, they have both met this test very 

well. The Federal service while not getting enough top quality 

talent to meet our needs is getting its fair share of what is 

available . 



The specific application of these methods to teacher re- 
cruitment and' selection for a large metropolitan school system is a 
matter which obviously needs considerable careful study. Without 
knowing more about the way in which such a system operates, it 
would be presumptuous of me to make specific proposals as to the 
best combination of selection methods to be used for teacher re- 
cruitment. I feel strongly, however, that any thorough- going study 
designed to produce the most effective selection system has to ap- 
proach the problem with a hard-hitting and searching analysis which 
gives full consideration to some of the possibilities suggested by 
this paper. 






o 

ERIC 



Impact of Critical Shortages on Teacher 
Recruitment and Selection Policies 

William B . Brown 

Los Angeles City Schools 



Purpose of This Paper 

This is a brief statement relating how the Los Angeles 
City Schools recently encountered major difficulties in teacher re- 
cruitment which, if not solved, would have resulted in seriously 
adverse effects on the pupils and the educational program, and how 
well-established policies and procedures were adjusted in the face 
of the crisis to meet staffing shortages. I hope the experiences 
which I will report here and our tentative conclusions will be 
pertinent to the objectives of this conference. 

The questions we are seeking to answer are: What are the 

best steps to take to meet an anticipated, crippling shortage? 

What measures will bring the best results for the schools in the 
short period of time available? The answers vary considerably from 
district to district, but we hope there are some common elements on 
which mutual discussion will be beneficial. 

Finally, this paper seeks to serve as an argument against 
too much rigidity in selection policy. Policy must be broad enough 
and flexible enough to meet a great variety of teacher supply- 
demand conditions— general oversupply or general undersupply; over- 
supply in some secondary fields and undersupply at the elementary 
level; oversupply in one part of the district and undersupply in 
another; keen competition with smaller, more attractive districts; 
or unexpected and sudden increases in demand as a result of enact- 
ment of specially funded programs with local, state, federal, or 
foundation resources. All of these require practical, vigorous, 
adaptable policies which will produce the greatest possible recruit* 
ment results in the limited time which is generally available. 

The critical elementary teacher shortage which the Los 
Angeles City Schools faced this year is an excellent illustration 
of the type of staffing problem encountered and the procedures in- 
stituted to meet the situation. The findings and conclusions are 
based upon our experience with this particular shortage, and with 
other shortages experienced since World War II. These policies and 
procedures are fully developed in personnel guides. They are ready 
for immediate implementation, and are sufficiently flexible so that 
adjustments can be made in terms of a particular emergency and a 
particular kind of teacher which may be in short supply. 



U8 



mmmm 






* ** -* * **** ik'letf. W*{ 






| 



Los Angeles City Schools — An Overview 

A little background information regarding the Los Angeles 
City Schools, the second largest school system in the nation, may 
be in order here. 

The district is responsible for the education of 13.5 per 
cent of California's elementary school pupils and 17*3 per cent of 
those in the junior and senior high schools. When the current 
school year started in September, Los Angeles had a total of 367,000 
pupils in U28 elementary schools, 136,000 students in 71 junior high 
schools, and 128,000 in 53 senior high schools. Our total enroll- 
ment, including adult schools and junior colleges, is approximately 
79^,000, and is expected to go over the 800,000 mark next February. 

Our teaching staff now totals approximately 31,000 in- 
cluding adult education and junior colleges. Our non-certificated 
staff now totals in the neighborhood of 20,000. 

Unlike most other major metropolitan school districts 
which have large student populations and serve more limited geo- 
graphic areas, the Los Angeles district serves a sprawling area of 
almost 800 square miles. Its service area is roughly twice as 
large as the City of Los Angeles and includes, aside from the City 
of Los Angeles, eleven incorporated municipalities and large sec- 
tions of unincorporated County territory. On a north-south axis, 
one can travel approximately 60 miles from the northernmost school 
in the San Fernando Valley to the southernmost school overlooking 
the Los Angeles Harbor in San Pedro, and never leave the district. 

The Los Angeles district serves areas of wealth, but it 
also serves major areas of poverty. Almost 200 of the 599 schools 
are located in "disadvantaged areas." 

The implications and relationships of these characteris- 
tics to recruitment are many. I mention but one. This year, a 
surplus of teachers wished to work in suburban San Fernando Valley, 
a largely middle class, Caucasian area, whereas there were teacher 
shortages in some other areas such as the central city. 

Teacher Recruitment Situation in Los Angeles 

Since World War II, Los Angeles has experienced a short- 
age of teachers of one type or another almost every year. A seri- 
ous elementary shortage existed for a number of years during and 
immediately after World War II. Then, there was a gradual improve- 
ment in the supply at that level between 1955 and i960, while, dur- 
ing this same period, shortages began to develop at the secondary 
level. The latter have intensified and continued for most of the 
past seven or eight years. 
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September, 196 $, was the first year since World War II in 
which Los Angeles did not encounter major shortages at the elemen- 
tary or secondary level except, of course, in mathematics and one 
;,vr two technical fields. This favorable situation was due to two 
major factors, an 11 per cent increase in district starting sala- 
ries for teachers, which brought the minimum annual pay to $6,120, 
close to the highest in the state; and a major change in State 
licensing requirements under a new credential structure. The large 
number of candidates was directly traced to the efforts of young 
people to qualify under the old credential structure before the new 
requirements went into effect. Had it not been for these two fac- 
tors, the shortages experienced during preceding years would have 
existed in September, 1 965 . So for most of the period from World 
War II to the present, Los Angeles has had the challenge and re- 
sponsibility of meeting teacher shortages of varying degrees of 
seriousness and type. The problem is not unusual in our city. 
Rather, it is a normal situation which must be continually faced by 
the personnel staff. 

The Los Angeles district has shown a consistent annual 
growth, following the end of World War II, of from four to six per 
cent. Since the early 19 S 0 1 s, we have been faced with the task of 
recruiting an average of 3,000 teachers annually to meet growth and 
replacement needs. But, in preparing our recruitment program for 
September of the 1966-67 school year, we were certain of one over- 
riding factor— this was to be no "ordinary’* recruiting year, not 
that there ever is an "ordinary" recruiting year. 

For one thing, our operating divisions gave us the task 
of recruiting a total cf lj.,500 new teachers for this school year. 
The unusually large demand was caused by the projected implementa- 
tion of several programs of a compensatory education nature, to be 
funded out of state and federal resources . 

Personnel staffs throughout the country are all too 
familiar with the problems incident to participation in state and 
federally funded special programs— ESEA and the Economic Opportun- 
ity Act, for example. These programs have added a new dimension to 
planning for school districts involved, generally because of delays 
in approval of legislation but particularly because of last minute 
"under-the-wire" administrative approvals from the funding source. 

Aside from the uncertainties attributable to the imple- 
mentation of special programs, Los Angeles recruiters were being 
asked to find the needed teachers in the face of a severe shortage 
of elementary teachers, the worst in many years, and it is to this 
problem that the remainder of this paper will be directed. 
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Elementary Teacher Shortage and Steps Taken to Meet It 



The serious shortage of elementary teachers anticipated at 
the start of the spring recruitment drive was based primarily upon a 
33 per cent drop in the number of elementary student teachers 
trained at the 12 colleges and universities in the Los Angeles area, 
which have been the source of two-thirds of the new elementary 
teachers for the District. Thus, this serious drop in enrollment, 
combined with a doubling of demand, pointed very clearly to a criti- 
cal problem facing the Los Angeles City Schools. 

Other factors which have had their effect on the number 
of potential candidates available can be traced to the implementa- 
tion of new credentialing legislation. These include: 

1. Lack of adequate, up-to-date information in out-of-state 
institutions concerning California credential require- 
ments , 

2. Requirement of a fifth year of college and a subject 
matter major for full credential status, 

3. Difficulties out-of-state recruits have in obtaining 
graduate standing in local institutions and in obtaining 
needed courses to meet the requirement of a fifth year. 

Our normal needs for growth and replacement have been ap- 
proximately 900 elementary teachers for the fall semester of each 
year. This spring we were faced with the recruitment of an addi- 
tional 900 teachers to meet the following needs for September: 

1. Implementation of legislation which reduced class norms 
to 31 in grades one through three... 293 additional 
teachers, 

2. Additional kindergarten teachers to staff a district in- 
crease in the kindergarten program. ..^U additional teach- 
ers, 

3 . Additional teachers for Elementary and Secondary Educa- 
tion Act programs ... 92 additional teachers (600 addi- 
tional had been employed the previous February because of 
ESEA). 

U. Implementation of new state legislation which reduced the 
pupil- teacher ratio to 23> to 1 in disadvantaged areas... 
U62 additional teachers. 



51 



Intensified spring re cruitment program: 
were taken by the district stiiT to meet tHe 
teacher shortage: 



A number of steps 
anticipated elementary 



1. Recruitment trips to other parts of the United States 

were more than doubled over those of the preceding year; 
00 campuses were visited this past spring as against 80 



• Both local campus and out-of-state recruitment trips were 
scheduled at earlier dates than in preceding years to 
place the district in a more advantageous position. 

3. The time for processing out-of-state recruits and making 
contract offers was cut in half over the preceding year. 
This time is now five to ten days from date of interview, 
depending upon when transcripts are received at the 
Personnel Division. 

It. Credential service to new recruits was substantially in- 
creased, including information and explanation of new 
regulations, individual counseling, and evaluation of 
transcripts.. This service was necessary to overcome con- 
fusion and misunderstanding concerning the new credential 
structure. 



The results of the, spring rocruitment drive for 
teachers were disappointing but riotT unexpected : 



new elementary 



1. Only ItlJ were recruited at local campuses, a drop from 
600 the previous year. This 31 per cent reduction tal- 
lied closely with the drop in student teacher enrollments 
mentioned previously. 



2. The number of out-of-state recruits showed a similar drop 
over the preceding year in spite of the expanded recruit- 
ment program. Only 1^0 elementary recruits were obtained 

£f° m ° ut -° f -? tate sources this past spring compared with 
2k0 the previous year. This 37*5 per cent reduction par- 
a II e l e ° the reduction experienced by other Southern 
California school districts. 



An additional 2f>0 new teachers were obtained from three 
other sources: former teachers returning to the District j approved 

secondary teachers in surplus fields, such as social studies, who 
were found to be qualified for elementary teaching service: and a 
limited number of teachers from other districts in the state. 



The spring 
only 81$ of the 1,800 



recruitment drive thus had netted the district 
new elementary teachers expected to be needed 
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for the start of the fall semester. The personnel staff concluded 
that a critical shortage would exist this September unless emer- 
gency action were taken. Two steps were therefore taken: 

1. A request ,was made to the State Board of Education for 
authorization for Los Angeles to use the provisional cre- 
dential. (This credential in California is granted on an 
individual district basis when application to the State 
Board shows existence of a shortage and the fact that 
proper steps have been taken to meet the shortage. The 
credential requires the possession of a baccalaureate 
degree only; no teacher training is required . ) 

2. Plans were made for an intensive summer community recruit- 
ment drive to secure a minimum of 600 additional qualified 
teachers who might be employed on regular or provisional 
credentials • This drive was undertaken during August and 
early September. (Partial relief from the overall goal of 
1,800 new teachers needed for the September opening of 
school resulted from a delay in implementation of one of 
the new state laws. This reduced the number needed by 

hoo.) 

The intensified summer recruitment drive was carried out 
with excellent support from newspapers, television, and radio. A 
community-wide appeal was made for qualified persons, housewives, 
and persons in other fields of work to respond to the urgent need 
for additional elementary teachers for the district. A special ap- 
peal was made to former teachers and to teachers on long-term leaves 
of absence to return to service this fall because of the shortage. 

The response far exceeded expectations and was better 
than the response to similar drives held in preceding years : 900 

applications were received, and 600 of the candidates were approved 
for contract assignment* 300 were employed on regular credentials, 
and 300 on provisional credentials. As a result of this drive, the 
elementary schools of the district started the fall semester with 
an adequate supply of new teachers. 

For the second semester of the current school year, which 
starts in February, many more additional new elementary teachers 
will be required, 3£0 for growth and replacement, and U60 to imple- 
ment the delayed specially funded state program for disadvantaged 
area schools. Once again, the Personnel Division will be confronted 
with the need to conduct a major recruitment program in the face of 
serious shortage at this level. Plans will be similar to those car- 
ried out during the summer months and will be conducted with every 
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expectation of finding all or a major part of the new teachers 
needed . 

The qualities we seek in all new teachers are: 

!• Strong interest in boys and girls, 

2. Knowledge of educational principles and practices, 

3. Knowledge of subject matter, 

U. Command of English fundamentals, 

5. Good scholarship, 

6. Ability to get along with others, 

7 • Appropriate appearance, 

8. Pleasing voice and clear speech, 

9. Ability to present ideas, 

10. Alertness and maturity of judgment. 

Initial employment screening procedures are exactly the 
same for all teachers — regularly credentialed or provisionally cre- 
dentialed. It is emphasized that all of the 300 teachers hired on 
provisional credentials were required to meet the same district 
standards as regularly credentialed teachers approved on a condi- 
tional one-year contract basis, including: 

1. A baccalaureate degree, 

2 . A strong college academic record , 

3. Strong personal qualifications for teaching, 

k» Passing of two objective tests, 

a. English usage 

b. Principles and practices of elementary education. 

The only difference between these two types of teachers is 
in the credential requirement. The regular teacher entering service 
on a four-year or partial fulfillment credential basis must have com- 
pleted student teaching or have had equivalent teaching experience. 
The provisional teacher does not have to meet this requirement. To 
offset this lack, the Los Angeles City Schools, whenever possible, 
require all new provisional teachers to undergo an intensive summer 
training, in the district, of not less than two weeks, and to serve 
the first semes tex* under the close guidance and supervision of a 
master teacher. It is preferred that all new teachers successfully 
complete a full student teaching assignment before starting service. 
However, in an emergency such as the present one, when this is not 
possible for all new teachers, district pre-service and in-service 
training programs can provide much of the necessary technical 
preparation. 

A seven-point district training program for new elementary 
provisional teachers includes the following: 

& 




1. Those hired prior to the opening of elementary school sum- 
mer classes are required to observe and participate in 
these classes for a minimum of two weeks. 

2. Those hired after the close of summer programs are given 
special orientation in the area offices. 

3. All provisional teachers spend a minimum of tw- 11 days 
in area induction meetings with supervisory per^nnel 
prior to the opening of school. 

U. Attendance in a semester-long in-service training course 
is required. This provides orientation to subject fields, 
methodology and curriculum of the Los Angeles schools. 

A permanent regular teacher :.s assigned full time as 
teacher advisor to each ten provisional teachers. The 
teacher advisor works in the classroom with the new 
teachers and provides conference time for assistance with 
problems . 

6. Each area supervisory staff devotes special attention to 
the needs of provisional teachers . 

7 . Principals give additional special assistance and 
guidance to the provisional teachers. 

Current Problems for Recruitment Review 

Each year new ideas and innovations are evolved to help 
meet critical staffing problems. The accumulation and use of this 
experience enable a district to face future shortages with consid- 
erable confidence. 

It has become axiomatic, for instance, that policies must 
be readily adjustable to meet emergencies, no matter how short the 
warning. Also, selection procedures must be considered within the 
context of the recruitment function, for there is no value in hav- 
ing sound selection procedures if no candidates will apply for em- 
ployment . 



In "tooling” up to meet the year-by-year needs for new 
teachers, it is important that recruitment planning and research 
provide guidelines for future action. There is often insufficient 
study and planning behind emergency recruitment programs. There 
are a number of basic questions to which we need to give increasing 
attention, if planning is to be more effective. Some of these are 
as follows: 



1. Has the job of the teacher changed significantly in re- 
cent years? 

2. To what extent is discipline becoming a more serious prob- 
lem than before and what implications does this have for 
the recruitment selection process? 

3 . Have criteria and specifications for selection been 
realistically revised in recognition of any important 
changes in the work of teachers? 

U* Have teacher training institutions been effectively in- 
formed of changes in the teacher* s job? 

Has teacher training been modified to meet the changes in 
the work of the teacher? 

6. Have these institutions recognized the great importance 
of preparing teachers for service in economically, educa- 
tionally deprived, disadvantaged, or culturally n dif- 
ferent” areas, and are the teacher trainees assigned to 
take at least part of their student teaching in poverty 
area schools? 

7. Have the colleges informed school districts realistically 
in their appraisal reports concerning student teaching 
success in handling pupils in poverty areas? 

8. Has recruitment been adjusted to meet changes in the na- 
ture of the teaching job? 

9. Is a prime recruitment objective the selection of teachers 
who are qualified to serve in all types of situations in 
urban districts, or can the criteria be specialized to 
meet varying community needs? 

10. Are all possible school and community resources being 
utilized for assistance in this recruitment process? 

3_i . Are we learning about and profiting from the influences 
which lead teachers to select large urban centers and to 
make a life career of teaching; in these centers; con- 
versely, are attempts being made to ascertain factors 
which keep prospective teachers away from urban centers? 

12 . Have urban districts learned how to utilize recruitment 

to improve the image of the large city as a fine place in 
which to teach and live? 

Need for more adequate supply-demand data: With current means 
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of communication and data processing, monthly reports on both sur- 
plus and shortages should be provided t"> school districts on na- 
tional and state levels. In Los Angeles, and in California general- 
ly, the current elementary teacher shortage could have been foreseen 
if all the pertinent data had been assembled and compiled in late 

1965. 



Events during the past year have made available supply- 
demand information completely inadequate. The war in Viet Nam with 
the attendant effect of the draft on teachers and prospective 
teachers, the civil rights problems in many of our cities, the 
specially funded programs to aid deprived areas — these and other 
influences are not readily translated into technical data. But 
translated they must be, and the data must be disseminated widely 
so that school districts and teacher training institutions may plan 
in advance to meet shortages most effectively. 

Conclusions Concerning Recruitment Policies 



Teacher shortages may not be as permanent as death and 
taxes, but, at least for the foreseeable future, recruiting of ade- 
quate numbers of well-trained teachers will be a perennial issue 
for personnel staffs of urban school districts. 

The time has long passed when those' of us in the person- 
nel offices of big city school systems, particularly those cities 
experiencing periods of growth, can afford to M sit back" and wait 
for fully-trained teaching candidates to flock to our personnel 
offices . 



Rather, we must formulate an aggressive, flexible, and 
comprehensive policy, which should be based upon the following 
facts : 

1. Close working relationships between public schools, 
teacher training institutions, and the State Department 
of Education are essential to meet shortage problems. 

The greater the shortage, the more cooperative effort on 
an immediate day-to-day basis is required. A team ap- 
proach is essential in coordinating credential policies, 
teacher training policies, district selection practices, 
and school staff requirements. 

2. A large and excellent reservoir of potentially fine 
teachers exists in metropolitan areas such as Los Angeles, 
which can be drawn upon in times of critical need. This 
reservoir is made up of college graduates, housewives, 
and persons in other fields of work, who have excellent 
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scholastic records in college, the personal qualifica- 
tions, and the background of experience for working ef- 
fectively with children. This reservoir should be tapped 
without hesitation when it has been determined that the 
supply of regularly credentialed teachers is exhausted. 

3. Television, newspapers, and radio stations can give 'very 
valuable help to a school district in a large metropoli- 
tan area, in recruiting needed new teachers from the 
ranks of housewives, recent college graduates, and per- 
sons in professional fields other than teaching. 

Ij.. Teacher salaries must be kept highly favorable. The. 

minimum salary must be the highest or close to the high- 
est in the state. This is essential if the largest city 
area and the largest school district are to compete ef- 
fectively with smaller and more attractive suburban 
school systems for the limited supply of new teachers. 

The well established screening procedures for the employ- 
ment of regular probationar teachers cannot be allowed, 
to interfere with the recruitment process in. meeting crit 
ical shortages. At the same time, an effective, stream- 
lined screening procedure must be utilized for initial 
employment on a contract basis to assure that district 
standards will be met. To state the matter bluntly, get 
the qualified recruit on the job first, then examine him 
closely for career status in the system. An important 
part of the screening procedure is the requirement . that 
the formal examination must be passed during the first 
year of service. 

6. The avenues of entry and recruitment contact must be 
manifold : 



a. District employment offices located at con- 
venient places within the school system, not 
restricted to one central location, 

b. Interviews with recruits on all local campuses 
of major colleges and universities, 

c. Out-of-state interviews at recruitment centers 
and on college and university campuses in 
various parts of the nation. 

Such recruitment requires a minimum staff for year-round 
operation, and amplified staff for peak recruitment dur- 
ing the spring and summer months. 
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7. Employment standards should not be lowered even in the 
face of serious shortage. For our district, only one ex- 
ception to this basic policy is made; that is, the meet- 
ing of the student teaching requirement. High selection 
standards are essential to preserve the educational pro- 
gram of the district. Recruitment must be intensified, 
salaries may need to be raised, processing of teachers 
may need to be streamlined , and pre-service and in- 
service help to new provisionals may need to be expanded; 
but under no circumstances should personal and academic 
standards be lowered. 

8. Technical credential requirements, which may be very sound 
when the supply-demand balance is good, should n °t be al- 
lowed to stand in the way of recruitment in time of short- 
age. Temporary waiving of such requirements, however, 
does not warrant a district's lowering its standards in 

any way. 

9. Resources of the supervisory and teaching staffs should 
be utilized to provide training and guidance for provi- 
sional teachers needed in an emergency. Lack of teacher 
training, including student teaching, is not an insur- 
mountable obstacle . 

10. Recruitment, above all, needs to be honest. If shortages 
exist only in disadvantaged areas, the community should 
be told this as frankly as possible. Recruitment should 
be undertaken directly and specifically to meet this 
shortage, seeking out the kinds of persons who not only 
have strong teaching qualifications, but also genuine 
interest in teaching in these areas of our cities. 



Looking Ahead 

1. Much more needs to be known concerning essential factors 
in successful recruiting of well qualified teachers for 
large urban districts. The entire recruitment function 
warrants greater study and research than it has received 
to date. Also, more research is needed concerning the 
most effective ways to screen and process recruits quick- 
ly so that they will respond to offers of employment from 
the urban district. 

2. Much better state and national information on the teacher 
supply-demand situation is required. Speedier and more 
frequent checking, both with colleges and public school 
systems, are needed. The current elementary shortage is 
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a classic example. A telephone call to the 10 or ljp 
larger districts in the country in late spring or early 
summer of this year would have revealed that most had, or 
were anticipating, a serious shortage. The hurricane 
flag was up, the barometer was steadily falling, the 
supply-demand relationship was far out of balance* but it 
was »' statistics” as usual. The supply-demand studies had 
been closed for the year. Clearly, a major overhaul and 
up-dating of our state and national information m this 

area is essential. 



3. Large urban districts must have the necessary freedom to 
solve their own recruitment problems. Too many outside 
agencies are interfering. Those who are closest to the 
recruitment and selection of teachers and who live with 
the problems the year round and have them in sharpest fo- 
cus, are the ones to take action and get the results. 
State and federal financial assistance is of increasing 
importance, but interference through technical credential 
regulations, such as exists in some of our states, is be- 
coming a serious obstacle to achieving necessary recruit- 
ment results. There is a need for a study of the forces 
at work at the state level, which often become major . 
problems for local school districts in carrying ou . eir 
st af fing responsibilities. Far too much staff time is 
now being spent in studying proposed new state regula- 
tions for support or opposition, understanding them after 
they are adopted, and making necessary adjustments. If 
local districts are given greater freedom and additional 
resources, the staffing problems of our large urban 
school systems will be effectively and promptly solved. 



The task ahead will require our best efforts. One of the most 
valuable ways to meet it is to have the kind of pooling of experi- 
ence among^he great cities of the United States which this confer- 

ence provides. 
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Desirable Policies and Procedures for Teacher Selection 

George B. Redfern 

V 

American Association of School Administrators 



That teacher selection procedures in public schools vary 
widely is an understatement of considerable magnitude. This vari- 
ability is to be expected since the control of public schools in 
this country is so highly decentralized and local in nature. 

In general, however, selection processes fall into two 
broad categories. One places a premium upon written and oral ex- 
amination procedures; the other puts emphasis upon other less pre- 
cise criteria and relies more upon the oral interview. At the risk 
of oversimplification, the former may be characterized by the term 
objectivity; the latter, subjectivity. 

That one process is manifestly superior and more effec- 
tive than the other has been argued for many years. During the 
early 1950' s the proponents of "both schools of thought" used the 
agenda of annual meetings of the American Association of Examiners 
and Administrators of Educational Personnel 1 to debate the issue 
of examination-oriented selection procedures versus non- examination 
techniques. Neither side quite convinced the other of the superi- 
ority of its point of view. The great debate ended mere in stale- 
mate than in victory. 

The use of well-designed, time-tested and carefully 
validated examinations in teacher selection in some school systems 
is justified on the grounds of historical precedent, legal require- 
ment, urgent necessity or some other compelling reason. At the same 
time, other systems use different procedures, justifying their use 
on equally logical grounds. The point is that school systems are at 
different stages in time and circumstance in their developmental 
growth. They reflect unique local requirements and conditions. 

They conform to specific demands made of them. 

The assumption is sometimes made that examination-oriented 
selection procedures are preferable, that they are more "scientific” 
and more effective in selecting good teachers than non- examination 
techniques . 



Since 19J>9, called American Association of School Personnel 
Administrators . 
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If this assumption is valid, then it seems to follow that 
other systems should adopt more examination- type procedures. Why 
do they not do so? Is it the result of unenlightenment? Is it due 
to lack of professional resources and skilled know-how? Is there 
an insufficiency of commitment to sound personnel administration? 

Or is it due to other plausible reasons? 

It is logical also to raise the question as to whether or 
not it is possible to increase the objectivity of so-called subjec- 
tive selection procedures without, however, completely adopting 
specific examination instruments. 

This paper is addressed to the consideration of the poli- 
cies and procedures of teacher selection in the Cincinnati Public 
Schools where I was, until just recently. Assistant Superintendent 
of Administration and Personnel. Written and oral examinations are 
not employed in Cincinnati but care is taken to compile "subjective 
data" in a manner that will yield eligibility scores which make it 
possible to rank applicants so that those with higher scores can be 
employed before those with lower scores. Cut-off points are estab- 
lished to eliminate the marginal and weak candidates. 

The purpose of this paper, then, is to show how so-called 
subjective selection procedures may be made objective, eligibility 
scores established, candidates ranked and selection may be made in 
terms of relative rankings. 

Certification - Written and oral examinations are more 
necessary when qualifications for certification are being ascer- 
tained. On the other hand, however, many state departments of 
education grant certificates on the basis of approved patterns of 
preparation without requiring the candidate to take qualifying 
examinations . 

In those instances, therefore, where state departments of 
education certificate teachers, it is possible to assume that some 
degree of selectivity has already taken place first, during the 
teacher education process and secondly, at the time the certificate 
is granted. This means, consequently, that the employing school 
system has some assurance that properly certificated applicants 
with sound preparation from accredited teacher education institu- 
tions have already been screened and evaluated to some degree. 

The relevance of the above observation is that school 
systems not responsible for examining for certification have cer- 
tain assurances about the relative qualifications of candidates 
without administering comprehensive testing procedures. Certifi- 
cation and credentials provide varying degrees of the necessary 
evidence for determining eligibility for appointment. 
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Preparation - While there is still considerable variation 
in the quality of preparation in different teacher education insti- 
tutions, higher and higher standards of preparation are required in 
an increasing number of colleges and universities. NCATE^ approved 
institutions have particularly high preparational requirements. 
Candidates from these institutions have been subjected to many meas- 
ures of selectivity. This gives further assurance and removes some 
of the necessity to assume that total selection is the responsibil- 
ity of the employing school system. 

The Cincinnati Selection Plan 

After applicants have been identified through recruitment 
efforts, a careful selection process is used to determine those 
best qualified for employment. The employment policy of the school 
system is to employ from the available applicants those best quali- 
fied to fill the positions which are open. 

Procedure - Step 1 consists of processing the application. 
This involves inspecting the application form to see if it is com- 
pletely and properly filled out . Care is taken to make sure that 
the record of past employment is complete, without gaps. A com- 
plete transcript of college credits is required. A copy of the 
teaching certificate is also required if the applicant holds one. 

If a beginning teacher, the applicant is required to show evidence 
of having fulfilled the requirements necessary to obtain the cer- 
tificate . 



Step 2 involves the processing of professional references. 
Evaluation forms are sent to all persons whom the applicant has 
given as professional references. Care is taken to ascertain if 
the names given represent persons who are in a position to give 
responsible, recent and relevant evaluations of the candidate’s 
qualifications . If this is not the case, contact is made with the 
applicant for additional references. All references are then 
evaluated as described later. 

Step 3 is the analysis of the application. The applica- 
tion, references, evidence of certification or eligibility for cer- 
tification are analyzed to determine if the minimum standards for 
employment have been met or can be met. 

Step U is the oral interview. If the candidate meets min- 
imum standards on the basis of academic preparation and professional 
references and if vacancies exist for which he is qualified, an oral 
interview is scheduled. 

^National Commission for the Accreditation of Teacher Education. 
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When the interview is held at the headquarters of the 
Board of Education, every effort is made to have at least two inter- 
viewers. One represents the personnel office; the other either 
someone from the department of instruction or administration. Fre- 
quently, a principal is invited to serve on the interviewing team. 



In campus interviews, the team interview is not always 
possible, although the practice of sending teams on campus inter- 
viewing trips is on the increase. In campus interviews, the oral 
interview takes place before the application is processed and cre- 
dentials are received. However, care is exercised to study the 
available records on file in the appointments' office of the 
training institution. 

»*• 

Four factors are stressed in the oral interview: 

(a) Scope and quality of training and preparation 

(b) Aptitude for teaching and working with children 

(c) Dedication and desire to teach 

(d) Character, personal fitness and mental health 

Determination of E ligibility - Upon completion of all 
application procedures, an eligibility grouping is determined for 
each candidate as follows: 

Priority Group A - This is the preferred group and all 
candidates in this group are considered for employment 
before candidates in all other priority groups are con- 
sidered . 

Priority Group B - This group is considered as satisfac- 
tory, meaning that minimum employment standards have been 
met. Those in this group are considered after all those 
in Group A have been employed or have rejected offers of 
employment . 

Priority Group C - These are the marginal candidates and 
are employable only in emergency situations. These ap- 
plicants usually are lacking in one or more of the basic 
requirements for full contract employment. 



Priority Group D — Persons in thus group are not employ— 
able. 

The following criteria are used to determine priority 
groupings. The first criterion is professional references. If an 
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teach^s U mIt r be onl^those administrators and supervisors who have 
observed teaching performance. 

In case of an odd number of references, the median ref - 
in case ui du thpre is an even number of 

erence (in terms of score) *. Dr iority groups, the appli- 

references equally divx e e Professional references are 

^ssi?ied S inSf four levell as follows: A-Outstanding, B-Strong, 

C-Acceptable; and D-Weak or below. 

The second criterion is the oral interview.^ If therefore 

three int egers g^en by^heTeprt 

interviewers, and the ratings ha ll be used. (The rationale 

sentative from the personnel the personnel 

and P on the aver- 

^gf t^ir a ratingsTavf s/own greater validity over the years in 
relation to success in teaching.) 

The four levels of the oral interview are: A-86 - 100 5 

B-80 - 85; 076 - 79; and D-75 or below. 

SfSfc « th, transcript *• f~ ' **“ 

following scale is used to assign pri 7 S jP dividing 

Z S 3 S Se^rr^total hunger of quality points. 
Overall eligibility groupings 

1 . The priority groupings for references, endemic 
record and the oral interview are converted into 

points . 



Priority Group 



A 

B 

C 

D 



Points 

3 

2 

1 

0 



2 The points for references, academic record and oral 
■ are multiplied as follows: 
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Points 



References 
Transcript 
Oral interview 



x 2 
x 3 
x $ 



3 . 



-The total points for references, academic record and 
the oral interview are added and an assignment is ma e 
to the proper priority group. 



Priority Group 



A 

B 

C 

D 



Total Points 

26-30 

16-2$ 

6-1$ 

$ or below 



Health Examination - The offer of a position to an appli- 
cant who qualifies for appointment on the basis of references, aca- 
demic record and oral interview is contingent upon Passage of a 
physical examination. This must be given by a Board of Ed ^ a ^° n 
nhvsician. the physician- in-charge being a psychiatrist. I 
individual fails the physical examination, the offer of a position 

is withdrawn. 

In some cases, late In the summer and due to scheduling 
problems it may not be possible to get the resultsofthe examina- 
tion until after the individual reports to his assignment. When 
this occurs, .it is under stood that the appointment will be limit 
to one year with contract not being renewable. 

Obviously, if the physical deficiency is of such a nature 
as to pose a risk to children, the individual is released and ? re- 
placement is obtained as quickly as possible upon receipt of 
results of the physical examination. 

Most of the physical examinations are given by general 
oractitioners but the results of the examinations are reviewed by 
the physician-psychiatrist in charge. Thus, if there are symptoms 
of mental health deficiencies, the personnel office may be alerted 
so that appropriate action may be taken. 

Periodic physical examinations are continued during the 
tenure of the teacher at intervals of three years. These may be a 
Board of Education expense, using the services of Board P h ysicians 
or the individual may use his own physician provided the examination 
is reported on forms provided by the Board. 

References - The three basic criteria in selection are the 



66 



ERIC 






references, academic record and the oral interview. It is recog- 
nized that, professional references vary in reliability. Some are 
relatively worthless; others are quite valuable. 

Greater reliability is achieved by requiring the refer- 
ences on specified forms which can be analyzed and scored as has 
been earlier described. References are given- the least weight among 
all the factors in recognition of their varying reliability, i.e., 

20 per cent of the total. 

Academic Record - Academic achievement, especially in 
student teaching, has proved to be a moderately reliable indicator 
of competence in teaching. There are exceptions to this generaliza- 
tion, however. Occasionally a candidate with excellent grades in 
college does not do well in teaching. On the other hand, one with 
average or slightly above average grades does very well in the class 

room. 



On the whole, however, the better the academic record, the 
more promising the prospects for success in the classroom. In 
weight, the academic record is given 30 per cent. 

Oral Interview - The oral interview is given the most 
weight in selection, 30 per cent. Care is taken to see that it is 
thorough and consistently done. Multiple interviews are preferable 
to single interviews. Of the three selection factors used, it tends 
to have the most consistent and predictive value in forecasting 
teaching success. 

No single criterion is adequate, however. The three fac- 
tors in combination do the best job. Mistakes occur, on occasion, 
regardless of the care and fidelity with which the selection process 
is carried out. Pressure, rush of time and volume of work at peak 
periods in teacher selection introduce error. The incidence of that 
error, however, can be kept small by. deliberate effort on the part 
of personnel administrators and examiners • 



Conclusion 

Much has been written concerning selection techniques and 
measures. Complete reliability and validity are difficult, regard- 
less of the type of procedures used. Insightful personnel adminis- 
trators and examiners can identify a number of relevant qualities 
that a good teacher must have. The problem is how to assess those 
qualities with certainty and consistency. This is the point at 
which difference of opinion exists and which is the concern of this 

conference . 

Pre- employment selection, at best, can only be relatively 
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successful. For this reason, it seems feasible to consider the pro- 
bationary period as a part of the selection process. During the 
first three years of service, especially the first or second, much 
reliable " evidence" may be obtained which can be used- to determine 
ultimate success in the classroom. 

Selection policies and procedures must be continuously 
studied and tested for validity and reliability to the extent that 
this is possible. Most school personnel administrators and examin- 
ers are quite conscious of this imperative. As promising techniques 
are revealed through exchanges of information in conferences of this 
type, each school system participating is enabled to re-evaluate and 
refine its own procedures. 

The great need is for more research and a greater exchange 
of information. There needs to be more dialogue among personnel ad- 
ministrators and examiners regarding the implications of this 
research and the data it reveals. 
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The Problem of Recruitment 
Albert Schiff 
Detroit Public Schools 

At the time of this writing, the author was awaiting the 
results of last minute frantic Labor Day week-end recruitment ef- 
forts of our personnel staff to enlist the aid of qualified teachers 
in the greater Detroit metropolitan area to return to teaching and 
in some measure alleviate the teacher shortage— which in mid-August 
was over 600 vacancies in our elementary and secondary schools . . We 
had just two weeks earlier made special appeals through the media of 
television, radio, and the press to enhance our efforts. 

Some solace was found v?hen he learned that other cities 
were in the same difficulty. The New York Times of September U, 
1966, reported that teacher scarcity in the public schools of our 
nation was the most critical in a decade. Its editorial of the same 
date stat'd that many factors had contributed to the scarcity. This 
was well known to school people throughout the country. But the 
editorial emphasized that educators ought to re-examine the su PP-}jf 
lines in the light of their broadened mission. In brief, it not@d 
that if present training facilities were numerically and qualita-. , 
tively inadequate, then they must be reorganized, reformed, and 
augmented by new types of institutions, probably with federal 
assistance . 

The gauntlet has been thrown to educators indicating that 
we should use the teacher shortage not as an excuse which besets, 
many school systems — but rather should signal the need. for a nation- 
wide analysis of teacher training and teacher utilization so. that 
a qq levels of education can be assured of a steady flow of high 
quality instructors. 

This is the crux of the entire area governing teacher se- 
lection and should pinpoint the direction we pursue for the future. 
For the most part, large city school systems set. up screening de- 
vices and techniques to "weed out" the less qualified and less de- 
sirable teacher candidates who have already graduated from teacher 
training institutions . Boards of examiners and. other personnel 
officials set up programs of testing, interviewing, examination of 
credentials, etc.— at great expenditure of funds— to insure that 
their school systems do not employ teachers who can give less than 
quality education to their students. This is deplorable especially 
at a time when staffs of this nature could be devoting theilr time 
and energy in working out newer and better ways and means of , 

selecting teachers for the .present, and future. ' ,'•••• - ‘ s 



69 









At a recent seminar devoted to high priority areas in 
school personnel administration^ — one of the speakers suggested that 
in the post-war years we haven’t been too concerned with selection 
because of the acute teacher shortage and predicted that this would 
soon change. In fact, he quoted a personnel director of one of the 
California areas who said he wasn’t quite sure that things would be 
any easier once there is an adequate supply of teachers. He felt he 
would find himself getting into more trouble when there are several 
teachers to choose from than when he was just lucky to get a posi- 
tion filled. I cannot share this optimistic view as it relates to 
large city school systems. In Detroit, for example, even if suffi- 
cient funds were available for the employment of teachers we need — 
it would be necessary for us to find approximately 2,000 teachers 
annually for the next several years to (1) maintain and expand cur- 
rent services, (2) provide for the normal leaves of absence, retire- 
ments, and resignations, (3) make provision for additional compensa- 
tory teaching staff in the lower socio-economic areas of the city, 

(I;) provide for the expansion of the Federal Programs which augment 
and enhance our school program. These reasons, coupled with the 
potential loss of many male students who are being classified to I-A 
under current Selective Service requirements, convinces me and many 
others that the normal channels of teacher education — e.g., the four- 
year accredited teacher- training institution will not be able to 
meet the needs of our large city school systems in the latter decade 
of the sixties and the coming seventies. 

Standards and Procedures for Teacher Selection 

Standards for school personnel administration were clearly 
delineated in a bulletin issued by the American Association of 
School Personnel Administrators in i960. This bulletin stated that 
whether selection is the outcome of a carefully designed inquiry and 
evaluation or is based upon less thoroughly developed processes, cer- 
tain facts should be known about a candidate prior to his initial 
appointment . 

1. Investigation and Screening - The amount of informa- 
tion available on each candidate during the screening 
process determines the completeness of the preliminary 
picture of his abilities and has to represent what is 



High Priority Areas in School Personnel Administra tion ; Bridging 
the Gap Between Research and Practice. Edited by'Harold E. Moore. 
Bureau of Educational Research and Services: Arizona State Uni- 

versity, 1 965 . 

'Standards for School Personnel Admi nistration . American Associa- 
tion of School Personnel Administrators. The Association, i960. 
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known about him. It becomes the foundation upon which 
a superintendent justifies his recommendation for ini- 
tial or for probationary appointment. 

a. The selection procedure is uniformly applied to 
every candidate prior to his, appointment . The 
selection procedure is essentially the same for 
all candidates. 

b. Prior to appointment to a position, information 
is on file showing the candidate^ standing in; 



(1) Subject area competence. 

(2) Cultural knowledge in such other areas as con- 
stitute the school curriculum, e.g., English, 
science, mathematics, social studies, and such 
related areas as foreign language, arts, and 
music . 

(3) Professional knowledge, including history of 
education, child psychology and development, 
educational psychology, and such related 
areas as curriculum, guidance, tests and 
measurements, and methods. 

(U) Intelligence, alertness, or reasoning ability. 

(£) Physical fitness. 

(6) Mental health. 

c Written evaluations or descriptive reports on his 
practice teaching or full-time teaching perform- 
ance have been reviewed. 



d. 



College records are on file, including transcripts, 
personnel folder, and record of college activities. 



e. Systematic and consistent means are used to assess 
»» personality." Independent opinions should be re- 
corded of at least three individuals as revealed 
through such sources as employment or performance 
records, college records of activities, and infor- 
mation independently solicited by the employer. 



f . Ranked order score sheets or eligibility lists 
where used are developed from objective examina- 
tions or objective evaluations. Scores are not 
ordinarily influenced by letters of reference. 



g. Score sheets or eligibility lists, where used, are 
made up of scores sufficiently reliable and valid 
to justify their use in promulgation of the ranked 
listing of candidates. 

h. If scores are used to represent performance or 
findings and if such scores are so combined and/or 
weighted as to show ranked order of candidates on 
score sheets or eligibility lists, such sheets or 
lists are strictly adhered to in the employment of 
candidates. 

2 . Interview - The term interview here refers to the pre- 
employment interview as distinguished from an informal 
or introductory interview. 

a. The employment interview is conducted by at least 
three fully certified and experienced staff mem- 
bers who are well acquainted with she procedures 
and problems of the profession, 

b. Members of the interview committee represent not 
only the subject area concerned, but significant 
related areas or professional concern as well. 

c. The interview is designed to enable the prospec- 
tive employer to learn those things about the 
candidate, such as personality, manner, and point 
of view, which may be only partly shown by his 
application and supplementary data. 

d. Instances where a single interviewer is authorized 
to employ an individual, the data collected for 
each applicant interviewed is inclusive and con- 
sistent. The interviewer has access to the pro- 
fessional opinion of at least two additional 
persons who know the candidate well. 

3. Appointment - Initial screening and final selection 
culminate in the superintendent's recommending the 
candidate for appointment. Only the Board of Educa- 
tion has the legal power to make a contract with the 
candidate . 

a. The Board of Education entrusts selection of teach- 
ing and nonteaching employees to the superintendent 
through his professional staff and does not enter 
into the professional processes of initial, proba- 
tionary, and final selection. 
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b. The public understands and the Board of Education 
assures that teachers are employed for their pro- 
fessional ability above all other considerations. 

c. Appointments are made only after all phases of the 
selection procedure are completed; they are then 
acted upon with reasonable expedition. 

d. Appointments are recommended by the superintendent 
and are consummated by the Board of Education. 

e. Appointments are made in writing and constitute 
binding contracts between the Board of Education 
and the educational staff member. 

f. The contractual agreement clearly indicates the 
nature of the appointment, whether probationary, 
regular, temporary, or substitute, and designates 
the term of service. 

g. The appointee receives a copy of the pertinent law, 
regulation, resolution, or appointment action 
governing his contractual situation. 

I would like to discuss briefly the method of selection of 
teaching personnel for the Detroit Public Schools. However, I do 
wish to call to your attention that this selection procedure is done 
after the college or university has had the student enrolled in 
their respective institutions anywhere from a period of three to 
five years. I include the selection procedure of Detroit only as an 
information item — which perhaps has some features currently in 
operation in other cities as well. 

The selection of teachers for the Detroit Public Schools 
is a continuous process involving the services of staff members from 
the schools, the Division for Improvement of Instruction, and the 
Personnel Division. The primary responsibility for teacher selec- 
tion rests with the Personnel Division. 

Beginning with the completion of application forms, the 
process follows step by step to a final judgment on the part of the 
superintendent or his Personnel Staff member as he examines evalua- 
tion summaries of candidates recommended for placement or non- 
placement upon the eligibility lists. 

Requirements for most teachers (other than emergency sub- 
stitutes, vocational day trade, and building trades apprentice 
teachers) are (l) a bachelor's degree from a college or university 
accredited by the North Central Association of Colleges and Secondary 
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Schools, or an equivalent agency, (2 ) a Mio ^8an Stata Provi8ional 
or Permanent (Elementary or Secondary) or Life ® d 

satisfactory college or university record, (k) a satisfactory 
of supervised or directed teaching, ($) status as ^ n ative o 
naturalized citizen of the United States, or an individual who has 
filed his declaration of intention to become a citizen, (6) a satis 
factory physical examination including a report on a chest Xray. 

The requirements for employment as a substitute teacher 
are the same as for a contract teacher, except tha seniors 
post-degree students who have completed their student ^aching sat- 
isfactorily may be employed as emergency substitutes while h y 
enrolled in courses necessary to complete work for their provis o 

certificates . 

Applicants for teaching are asked to file the following 
credentials! (1) Application for a Detooit Teachtag Position, 

(2) Transcripts of college records. It is the responsibility 
applicant to file his own transcripts. The applicant should request 
each college or university attended to send a transcrip . 
necessSy because, although credits transferred from one college to 
another L-e summarized on transcripts, the actual grades earned in 
each course ar“t furnished. Student copies of transcripts may be 
used in -Hry application, but official transcripts must be filed 
before” 1 employment^ (3) Recommendation folder from college placement 
bureau, if available. 

Applicants who do not meet the above requirements are 
notified of the specific reason they cannot be considered for em- 
ployment. As soon as enough credentials are received on w ^ c |\ 
base a judgment, applicants can be invited to participate in the 
selection process, and appointments for interviews are scheduled. 

Bachelor degree recipients and graduate students of local 
colleges and colleges accredited by the National Council for Accredita- 
tion of Teacher Education participate in a special selection P 
cedure and may be excused from the group tests provided they satisfy 

conditions shown below: 



Bachelor's Degree Recipients 

1. The candidate was in the "Upper 
half" of his graduating class. 



2. A mark of "B" or better was re 
ceived in directed teaching. 



Graduate Students 

1. At least l£ hours of 
graduate work with a mark 
of "B" or better. 

2. In undergraduate work, 
the candidate was in the 
"upper half" of a graduat- 
ing class from an accredited 
college or university. 
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3 . In undergraduate work, a 
mark of "B" or better was 
received in directed 
teaching . 

Applicants meeting these requirements and who are recom- 
mended by their colle’ge officials may be employed on the basis of a 
committee interview at the Personnel Division, at a Detroit school, 
or at a college campus. "Upper half" applicants are usually inter- 
viewed by a committee of two, inclusive of supervisors, administra- 
tors, and/or members of the Personnel Division. 

"Upper half 11 applicants not recommended for employment by 
an interviewing committee may be asked to participate in the regular 
selection process. 

Applicants participating in the regular selection process 
are required to take the National Teacher Examination or a local 
battery of tests. 

As a final step in the selection process, a meeting is 
scheduled with a Personnel Committee, usually consisting of a 
teacher and supervisor from the subject field; a principal, an as- 
sistant principal, or a consultant; a school social worker or a mem- 
ber of the staff of the Psychological Clinic; and a chairman from 
the Personnel Division. The selection committee reviews the college 
transcripts and recommendation folders, ratings, and test results. 
After the personal interview with the candidate, each member of the 
committee casts a secret, independent ballot rendering his judgment 
of the candidate. 

The recommendations of the personnel committees are sent 
to the Assistant Superintendent and/or Divisional Director of Per- 
sonnel for review and approval. His approved appointments are then 
presented to the Superintendent and then the Board of Education for 
official action. All candidates are notified of the results in 
writing . 



Successful candidates are informed that they are placed on 
the eligibility list contingent upon the approval of a Medical Exam- 
iner of the Board of Education and the filing of an official tran- 
script indicating a bachelor’s degree from a university or college 
accredited by the North Central Association of Colleges and Secondary 
Schools or an equivalent agency, and a Michigan State Provisional or 
Permanent (Elementary or Secondary) or Life Certificate. 

Candidates who meet the specific requirements for "upper 
half" candidates and are recommended by an interviewing committee 
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are approved for placement according to the date the applications 
S SscSpts are filed. Those who are approved by an interview- 
ing committee contingent upon satisfactory test results are placed 
onthe eligibility list according to the date of the test. Candi- 
dates whose test results are not entirely satisfactory meet a regu- 
lar teacher selection committee. If approved, the date for Place- 
ment on the eligibility list is the date of the regular selection 
meeting. Those 8 not classified as "upper half" candidates are placed 
on the eligibility list on the date they are approved by the teacher 
selection committee. 

Applicants who are not approved to participate in the se- 
lection process and applicants not recommended for employment by 
selection committees who believe their qualifications have not been 
properly evaluated may appeal to the Teacher Selection Review 
Committee. 



This now brings us to the core of the problem. Perhaps no 
profession has such a rate of turnover and mortality th^ the teach-- 
ing profession. It thus becomes apparent that we must work out tech 
niques of selection that would identify the candidate who has a com- 
mitment to teaching and subsequently remains in the field for a life 
time career. Selection procedures-as they pertain to personnel ad- 
ministrators-begin with the college senior. Although our selection 
nrocedures in Detroit are perhaps as adequate as most large c y 
school systems. I question the continuing need for teacher selection 

by personnel administrators of school systems whe ^.^ e t ^ e ^ 3 Ve 
process for future teachers should begin much earlier than 
college student approaching his senior year. 

In 196£ a special committee of the National Commission on 
Teacher Education and Professional Standards studied the area of the 
assignment and misassignment of American teachers . . I iscuss 
entire problem of staffing the schools with effective teachers, and 
came to the conclusion that this was a problem interlocked with 

several broader issues in American life. Thl ®.J?P° r * ^®^ ther " 
cussed (1) the continuing and increased competition between other 
professions and the schools for the services of talented individuals 
and competition among school districts themselves, (2) the Possi- 
bility of placing the right teacher with the right students at the 
right time when different school districts had conflicting social or 

intellectual goals, (3) the lag betweencurriculumofferedinthe 

colleges that prepares teachers and that offered in the public 
schools, (lj) the increasing demand for teachers and teacher organi- 
zations placing a greater burden upon administrators satisfy 
teacher's demands for proper assignments, (5) the question of whether 
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colleges and universities offer adequate programs to prepare 
teachers to meet conflicting demands of the great cities and the 
rural areas, (6) developments in organization and technology cre- 
ate new and different positions for both professionals and non- 
professionals in the schools, (7) the influence of consolidation 
and reorganization of rural school districts on the preparation 
and proper assignment of teachers, (8) the misassignment of teach- 
ers as a result of insufficient numbers or the inertia of those 
who are principally concerned with assignment. 3 

The report concludes that these and other major ques- 
tions do impinge and complicate the problem of staffing schools 
with effective personnel. 

In Detroit we often are hurled the challenge — n Why can’t 
we compete with the suburbs in terms of quality education, salary, 
etc.?” The sooner we face this problem realistically and indicate 
to all concerned that Detroit and other large cities are not sub- 
urban areas and that our teachers must reflect those ideals and 
characteristics which are necessary for effective teaching in a 
large, industrial, and cosmopolitan city— then and only then can 
we come to grips with certain basic procedures to be followed in 
the selection of our teachers for the future. 

In Detroit, over the past few years, the Personnel 
Division has sponsored clubs of future teachers ranging from the 
elementary through the junior high and the senior high schools. 

In 19^7, at the beginning of our efforts, only 12 high school 
clubs served 300 members. Now, over U,£00 pupils are participat- 
ing in l6l local school units. An everage of 2£ to 30 members is 
the usual size for these groups, although clubs range from as few 
as 8 members to 130 members. 

Primary emphasis in the daily activities of club members 
is service to school and community. This is common to all levels. 
Since 19^2 there has been a continuous increase in cadet teaching 
and tutoring which carries out a second purpose of the clubs, to 
familiarize students with the training, activities, and responsi- 
bilities of a teacher. 

All levels explore various fields in education and help 
students plan for college. At the elementary level, this planning 
is often the simple but inspiring visit to a metropolitan university 
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a goal for the future. At the secondary level such visits °rystal- 
ufe specific enrollment plans and concern entrance requirements, 
programs, costs j ajid f acili-tiss • 

The success of Detroit Future Teacher Clubs and those 
the country is apparent in reading the credentials of begin- 
teachers interviewed at the colleges. An increasing number 
list Future Teacher Club membership as an extra-curricular activity 

in the high school. 

Recruitment for the most part now begins with college 
seniors and student teachers. In view of shortages and other 
problems that face the large city this does not supply enough 
q ualif ied teachers to meet the demands of our society. Recruit- 
ment should start with great emphasis at high school, sophomor 
level This can be done through literature and speakers to sh 

the Lw role and responsibilities that teachers must face 
for the future. In our schools we have thousands of young people 
working VsZtkel Aides and/or are enrolled in work-study pro- 
grams ind other services which provide experiences that readily 
leST themselves to a good start for a future teacher. This couid 
be augmented by meetings sponsored by the personnel divis 
through conferences and city meetings ^ Jo enable servi 
change experiences in different types of schools. Such exper 
ences plus the raising of aspirations and expectations, would 
especially valuable with Negro youth. These experiences should 
™He confined only to the city but should be in the entire 

Metropolitan area. 

Even in our soci.ety— >as mobile as it is in terms of 
changes teaches make during the course of a lifetime--the great- 
est source of future teachers for any city rests with “ e 6 
school students in the classrooms of that city today. I hate 
use an old cliche but it is so true that- -"The teachers of tom 
row are in our high school classrooms today." 

The selection of prospective teachers then should be 
continued through college and the university level beginning at 
ttff^mTyfar. School systems should ^ovide a coordinator 
to work with teacher training colleges in their area. The aim 
should be to help the prospective teacher, suggest shortage areas, 
provide child-study opportunities through classroom visit^ions, 
teacher conferences, getting jobs as camp counselors, securing 
employment ^year-^ 

b^teacher^ducation and educational research divisions to the end 
that greater cooperation can be achieved between the school 
system and the teacher training institutions. 
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One of the outstanding developments which ought to be 



nurtured and developed in greater depth is the internship program 
as an integral part of the preparation for future teachers. The 
internship can be extraordinarily successful not only as a train- 
ing device, but also as a means of achieving a functional partner- 
ship in teacher education and public school systems. 

Michigan State University, in a bulletin describing its 
internship program, states that if the internship approach to 
teacher education is going to represent a significant step forward 
in teacher preparation, it should be incorporated in programs de- 
signed to meet certain criteria: 



1. Internship programs, to have a real impact, must be 
brought into the mainstream of teacher education. 
Unfortunately, most internship programs now operat- 
ing are on the periphery and include only an insig- 
nificant portion of the students preparing to 
become teachers . 

2. To continue as a growing edge in teacher education, 
such programs must be designed so the internship can 
be self-supporting. Many such programs presently 
operating are dependent upon foundation or other 
outside support. 

3. An internship program, to be worthy of the name, 
must provide careful and meaningful supervision of 
the intern by competent clinical supervisors. With- 
out such adequate supervision, an internship ex- 
perience can be sterile and lead to little profes- 
sional growth. 

h. The preparation of the intern prior to internship 
should approximate the preparation of any other be- 
ginning teacher. Xn the cooperative relationship 
with public school systems, teacher preparation in- 
stitutions must remember the first obligation of the 
schools is to the youngsters in the classroom, not 
to teachers -in- training. Therefore, interns must be 
we ll enough prepared at the time they begin teaching 
to provide high quality instruction. 



We should look favorably upon the use of Federal funds tc 
help us in our search for quality teachers for the future. There 
is a vast untapped source of college graduates who could be en- 
couraged to consider teaching as a career if we would provide the 
experience and training— while they are receiving reimbursement 
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during the period they are completing requirements for full 
certification. 

Special needs of a metropolitan system demand racially 
integrated school staffs and training of prospective teachers to 
meet the needs of "deprived” and/or "disadvantaged" youth. 

We call upon teacher training institutions not to recom- 
mend candidates for teaching about whom there are any questions of 
general intelligence, competence, personality, or other factors 
which preclude success in teaching. School personnel administra- 
tors have a right to assume these candidates are ready to "tackle" 
the rigors of a classroom given the aid of adequate supervisory 
and administrative staff. 

I believe large city school systems will need to modify 
rigid teacher recruitment and selection procedures to alleviate 
the teacher shortage by the adoption of temporary stop gap meas- 
ures— while still attacking the problem of effective long range 
programs . 
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Criteria: Problems in Validating Teacher Selection 

Policies and Procedures 

David G. Ryans 

University of Hawaii 



Nature of Criteria 

A good deal of confusion appears to exist with regard to 
just what we mean when the term criterion, or criteria, is em- 
ployed . A criterion is simply a standard or bench-mark used to 
provide a frame of reference for judging or evaluating something. 

It may be thought of as a model against which comparisons may be 
made. Usually criteria evolve from common agreement about accept- 
able standards — regulatory boards for insurance, public utilities, 
banking, contracting, and such operate with a set of agreed-upon 
standards (criteria) as a model. In many circumstances criteria 
are arbitrary and relative to values that are held to be import- 
ant by some particular group of persons at some particular time, 
and place. Indeed, the matter of "values” and "value systems" is 
basic to the consideration of criteria. 

In relation to teacher selection, just as opinions and 
preferences (values) of individuals vary with regard to the compe- 
tencies and behaviors expected of teachers, the criteria agains 
which teacher selection procedures should be compared will oiten 
vary (at least, in certain features) from community to community; 
and validity studies of teacher selection nearly always require 
replication adapted to the varying conditions. 

In taking this position that criteria are determined by 
value contexts that differ among schools and communities, I. am 
implying that the first step in the consideration of criteria 
against which to judge a teacher selection program must be to de- 
termine the expectations that are held locally with regard to 
teaching and teacher behavior. The extent to which there. may be 
concensus about major issues, the greater the assurance with which 
a school administration or its educational researchers may ap- 
proach the designation of criteria and their components, and the 
greater the possibility of conducting meaningful validity studies 
of teacher selection procedures. 



I shall return to the relationship between criteria and 
value systems later, but I first would like to comment further. on 
the nature of criteria— and the frequent neglect of and confusions 
about considerations relating to criteria. 
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Here I do not restrict my remarks to studies of teacher 
selection and research on teacher behavior; they are no more vul- 
nerable than a great deal of research in the behavioral -sc * ^nces 
where the problem of the dependent variable, or the criterion, has 
been neglected. From years of reading research reports and re- 
search proposals I conclude that many otherwise elegantly designed 
researches — well designed from the standpoint of sampling, control 
of the experimental variable and other independent variables, data 
analysis (and often involving real ingenuity of approach) — have, 
almost as an after-thought it seems settled upon some available in- 
strument, or perhaps hastily thrown together some test, inventory, 
or other observational technique without great regard to its valid- 
ity and reliability, proceeding on the assumption that such instru- 
ment satisfactorily reflected, and provided useful estimates of, 
the criterion behavior. This happens to be a pet peeve of mine — 
the fact that so many investigators seem to neglect or give too 
little attention to control of the dependent variable (criterion 
behavior) and that instead of considering this important problem 
from the very beginning of their research they appear to give it 
only cursory attention some time later in the investigation. I 
suspect in many such cases the researcher is introducing a source 
of Type II error; or when statistically significant relationships 
between experimental variables and the assumed dependent variable 
are obtained, the relationship really may be between the experi- 
mental variable and an unintentionally biased and unsatisfactory 
estimate of the criterion behavior. Although I feel my accusation 
is fairly generally applicable to research, it is one about which 
we teacher selection researchers certainly need to do some con- 
sidered soul-searching. 

Teacher Selection in Perspective 



It seems to me there is some similarity between what we 
are interested in when we plan teacher selection procedures and 
subsequently study their usefulness and the sorts of things a 
curriculum developer is concerned with. 

Typically, I believe, the planner of instructional tech- 
niques and course materials considers the ideal procedure to be 
followed as consisting of: (1) designation of course objectives, 

goals, and expectations; (2) breaking down the objectives into de- 
scriptions of (a) expected teacher behaviors and (b) expected pupil 
behaviors, i.e., the pupil behaviors it is intended the course or 
curriculum will help to nurture and develop; (3) planning and de- 
velopment of specific curricular materials and instructional tech- 
niques that are hypothesized to aid in developing the intended pupil 
behaviors; (h) selection of appropriate means of measuring the at- 
tainment of the behaviorally described objectives (Recall that any 
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one of a number of methods may be used — e.g., measurement of sam- 
ples of the pupil criterion behavior (samples of the expected pupil 
behaviors), measurement of aspects of teacher performance, measure- 
ment of teacher opinion about the efficacy of the program, citing 
of critical incidents, measurement of pupil behavior known to be 
related to the criterion behavior, measurement of pupil test re- 
sponse to verbally described situations related to the criterion 
behavior, etc.) ; (5) assembly of data (which may be in any of sev- 
eral kinds of units, scores, ratings, etc.) yielded by the measure- 
ment devices that were assumed to reflect attainment of the speci- 
fied objectives; and (6) evaluation of the course materials and/or 
instructional procedures by drawing inferences from the collected 
data about attainment of the course objectives. 

It may be laboring the obvious to spell out the closely 
related steps that are involved in the development of teacher se- 
lection procedures and their evaluation. Nevertheless I am going 
to describe what I believe to be a procedure that provides an ap- 
propriate rationale from which teacher selection should proceed. 
(Note that I consider this procedure to represent an »' ideal” one — 
one which often cannot be followed step-by-step in practice. Prac- 
tical considerations often demand that we skip early phases and 
proceed to set up teacher selection techniques on the basis of best 
available judgment — and I should not imply the selection procedures 
thus developed necessarily will be poor; they may be based upon 
substantial wisdom growing from experience, and upon testing them 
out they may be found to yield results that can indeed be shown to 
relate to valid criteria of teacher behavior, even though these 
criteria were not determined prior to the planning of the teaching 
program. I think we are getting the cart before the horse to de- 
velop selection procedures and then at some later time turn atten- 
tion to the criteria to which we think these procedures ought to 
relate — but sometimes this is the best we can do.) 

But let me get on to a statement of what I think we might 
agree would be a desirable way to proceed if we were operating with' 
in an ideal situation. It is a procedure that is fairly similar to 
the curriculum development paradigm I spelled-out a moment ago. I 
will refer to some ten steps or phases. 

(1) Selection and designation of general aspects of the 
value system framework of the school/community as they re- 
late to teacher behavior. I am referring here to the 
agreed-upon qualities that are desired, or expected, of 
teachers in a particular place and in particular kinds of 
teaching situations. (Note again that this process of 
arriving at criteria necessarily is subjective and a mat- 
ter of the values individuals or groups of individuals 
may possess in common. When we designate criteria we 
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proceed from a context of an accepted value system. We 
view teacher behavior in light of a set of attitudes, 
opinions, and viewpoints that reflect the sorts of teacher 
behavior we approve and prefer and also the kinds of be- 
havior we disapprove and find unacceptable. Value judg- 
ments and the value concepts and systems on which they are 
baaed grow out of highly personal biases, preferences, be- 
liefs , opinions , and attitudes we hold as individuals • 

To the extent any group of persons share in common cer- 
tain expectancies, preferences, or biases about teachers 
and teaching, criteria of teacher behavior may be defined 
for that particular group. Thus value systems concerning 
teaching, and criteria of teacher behavior, are likely to 
be relative rather than absolute. Although some "valued 
teacher behaviors" may be held in common by a large cross 
section of citizens and educators at a particular time, 
still other "valued behaviors" that must be taken into ac- 
count in specifying criteria may vary from one community 
or school to another.) 

(2) Identification of kinds of situations in which the 
agreed-upon "valued teacher behaviors" may occur — and in 
which they may be observed and assessed. 

(3) Operational description (i.e., description in terms 
of actual teacher behaviors) of the agreed-upon valued 
behaviors that are to comprise the criteria of teacher 
behavior . 

(U) Selection of methods of estimating the operationally 
(i.e., behaviorally) described valued behaviors. This is 
the problem of instrumentation relative to the criterion 
behavior and obtaining assessments of the criterion be- 
haviors. (Assessment relates to quantified, or quasi- 
quantified, description. When we make an assessment of 
some characteristic of some thing or some behavior, we 
are concerned with the degree to which that character- 
istic is manifest.) In assessing some aspect or charac- 
teristic of the criterion behavior of teachers we are 
•trying to estimate the extent to which that defined 
characteristic is manifest by some teacher. 

(£) Identification of observable properties of teacher 
classroom behavior that may be related to the specified 
operationally described criteria (i.e., the descriptive 
cataloguing of teacher characteristics and behaviors that 
occur in the classroom) . 
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(6) Development of selection instruments and procedures 
that are hypothesized to yield estimates that will re» 
fleet the operationally described teacher behaviors 
(criterion behaviors)— which, in turn, are assumed to 
reflect the value framework of the school and the 
community served. 

(7) Assembly of data yielded by the teacher selection 
instruments and procedures noted in Step 6 above. 

(8) Assembly of data yielded by the procedures used to 
estimate the criterion behaviors— Step 4 above. 



(9) Analysis of relationships between estimates of the 
behaviorally defined criterion behavior and the estimates 
of teacher characteristics used in the teacher selection 

procedure. 

(10) Evaluation of the teacher selection procedures by 
the drawing of inferences about the validity of those 
procedures for predicting the criterion behaviors 
designated in Steps 1 through 3 above. 



Common Confusions in Dealing with Aspect s of the Criterion Problem 

One of the reasons we have difficulty with the criterion 
problem is that we sometimes fail to distinguish between different 
aspects of what is involved. We have all heard "principals; rat- 
ines" referred to as a criterion of teacher behavior. X think it 
is not nit-picking to note that principals' ratings do not consti- 
tute a criterion or a description of a criterion behavior . ^hey 
are one kind of estimate, derived from one method of obtaining data 
thatTnay'Tunder some conditions perhaps, be related to some speci- 
fied aspect of the criterion behavior of teachers. 

Allow me to illustrate what I mean about confusion of 
terms with one or two examples . 

Let us suppose the value system in a particular school 
community expects its teachers to possess some degree of japabili y 
with respect to “classroom management." We may think oi tnis, 1 
believe, as a criterion of teacher behavior in that community. It 
is, of course, a very generalized and abstract description of cri- 
terion behavior at this stage. Before we can proceed to observe 
teachers with respect to their capabilities for classroom manage- 
ment we need to specify still further the kinds of behaviors that 
comprise this domain and we need to try to determine either (a) sam- 
ples of the criterion behavior that may in some manner be observed 
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and assessed and/or (b) known or assumed correlates of this cri- 
terion behavior that may be observed and assessed. As one example, 
of many that could be used, we might choose the teacher's response 
to a situation involving activities on the part of some pupil that 
interfered with the activities of his classmates in pursuit of the 
objectives of instruction. We are still talking about criterion 
behavior but we now have broken it down into a description (al- 
though still somewhat general) of a sample of the criterion be- 
havior. Now we might choose any one of several methods of estimat- 
ing the criterion behavior under consideration. And the method we 
would use would determine, at least to a large extent, the kinds of 
assessments or estimates of the specified criterion behavior we 
would obtain. We might choose to employ direct observation of 
teachers in the classroom by trained observers, and one of the 
kinds of estimates we might obtain by such a method would be ob- 
servers' ratings recorded on a scale representing qualitatively de- 
fined degrees of appropriate teacher behavior in the disciplinary 
situation referred to. Or, we might choose to use principals' re- 
call of teacher behavior in situations involving classroom manage- 
ment and this method might yield estimates in terms of some sort of 
ratings, rankings, etc. 

As a second possible example, suppose it was agreed that 
one aspect of the criterion behavior of a teacher should be his 
capability of communication of knowledge. In our attempt at be- 
havioral description of the criterion, one aspect of communication 
of knowledge might be determined to be the teacher's behavior in a 
situation involving the presentation and explanation of specified 
subject matter content. (This could be made still more specific — 
we might specify behavior that emphasized clarity and directness 
of presentation, or perhaps subject matter depth, or absence of ir- 
relevancies , etc.) In light of such criterion behavior we might 
resort to an observation method that involved the use of teacher 
examinations which would yield estimates of the teacher's knowledge 
and understanding of the specified subject matter. Or, we might, 
again resort to direct observation by trained observers and obtain 
ratings, frequency counts, or other kinds of estimates. Or, in- 
stead of employing a sample of the criterion behavior of the 
teacher per se, we might choose to view the criterion in terms of 
known or assumed correlates of ’"teacher communication of knowledge." 
In this case we might choose to measure pupil knowledge of particu- 
lar facts, principles, etc. that are assumed to be a product (at 
least in part) of the teacher's behavior in the communication of 
the specified knowledge. In this case as a method of estimating 
the criterion behavior we might elect to test pupil knowledge be- 
fore the teacher presentation and immediately after presentation, 
obtaining estimates of the differences in test-estimated pupil 
knowledge before and after exposure to the teacher's presentation; 
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or we might test pupil knowledge before the teacher presentation 
and again after some specified period of time — to obtain estimates 
of the extent to which the teacher communicated knowledge was re- 
tained by the pupil; or we might use a method of determining the 
success of the pupil in later situations for which the specified 
knowledge is presumed to be a necessary prerequisite— such proce- 
dure yielding estimates based on test scores and grades in subse- 
quent units of a course, in advanced courses, etc. 

I have used these examples to help distinguish aspects of 
the criterion problem that sometimes are confused when we discuss 
such matters. The methods of estimation and the estimates yielded 
by different methods of estimating criterion behavior should, I 
think, be clearly distinguished in our thinking from the descrip- 
tions of criteria against which teacher selection procedures may be 
evaluated . The criteria themselves are the behaviors of teachers 
that are held to be of value. And of particular importance to 
validity studies of selection procedures, we need to recognize 
identifiable behavior samples and known correlates of the "valued" 
behaviors that are accepted as the criteria of teacher competency. 

Some Considerations in the Designation and Estimati on of Criteria 

I would like now to note, at least in outline, some of 
the kinds of problems we must face in dealing with criteria. I 
will restrict my comments to two types of problems. One set of 
concerns has to do with (a) the validity of the description of 
samples and/or correlates of criterion behavior (i.e., the validi y 
of criterion descriptions in light of the value system involved) 
and (b) the generaliz ability of descriptions of criterion behavior. 
The other set of problems has to do with the validity and relia- 
bility of procedures that may be used for estimating specified 
criterion behaviors of teachers. I shall mention, but not discuss 
in any detail, three different concerns from the standpoint. of the 
validity of definitions and descriptions of criterion behavior in 

teaching . 



One such area of concern has to do with judgments about 
the dimensionality of the criterion behavior under consideration: 
(1) Is the criterion behavior uni- or multidimensional? (Needless 
to say we usually agree that teacher behavior involves a number of 
dimensions that interact in complex combinations.), (2) How do be- 
haviors that comprise important dimensions of teacher behavior ag- 
gregate— what are the behavior aggregates or patterns that really 
are relevant from the standpoint of teacher classroom behavior?. 
What is the relative importance of various dimensions of criterion 
behavior in teaching — and how should these be weighted in criterion 

description? 
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A second set of concerns having to do with validity of 
the criterion description regard the logical consistency and inter- 
relatedness of criterion dimensions--(l) How are the component 
dimensions of the criterion behavior patterned? (2) How do they 
overlap? 



Still another area of concern from the standpoint of 
validity of criterion description has to do with the sampling ade- 
quacy or representativeness of the criterion dimensions that are 
selected to reflect criterion behavior. This is essentially the 
problem of trying to arrive at a criterion description that is as 
free as possible of bias. A number of sources of criterion bias 
were described almost twenty years ago by Brogden and Taylor in 
their classic article on n The Theory and Classification of Crite- 
rion Bias" ( Educational and Psychological Measurement . 19£o, 10, 
l£9-l86.) I reviewed the bias problem drawing heavily upon the* in- 
sightful Brogden-Taylor treatment, and other considerations of the 
criterion, in The Journal of Genetic Psychology in 19^7 ("Notes on 
the Criterion Problem in Research, with Special Reference to the 
Study of Teacher Characteristics," J. Genetic Psychology, 19^7, 91, 
33-6l.) Since more detailed discussions exist, I will only remind 
here that in designating criteria against which to judge teacher 
selection techniques we must know how to recognize and must be con- 
stantly aware of conditions that may bias (and make useless) our 
criterion descriptions. I refer particularly to contamination bias, 
opportunity bias, experience bias, rating bias, deficiency (incom- 
pleteness) bias and distortion bias. 

With regard to the generaliz ability of the criterion def- 
inition and description, (and here I am speaking of the replicabil- 
ity of the criterion description under different circumstances) I 
again note the relation of criteria to value systems espoused by a 
group and the probable variation in adequate criterion descriptions 
ift different communities • We need be concerned whether criterion 
descriptions may vary from one kind of teaching situation to another 
for the same teacher and from one sample of teachers to another . 

Once the problem of criterion description has been faced, 
we must deal with considerations relative to choipe of the method 
or methods, of estimating the criterion behavior and the kinds of 
measurements or estimates that may be employed. Here again we are 
faced with the problems of validity and the reliability— this time, 
the validity and reliability of the t instruments and the data they 
yield with respect to the criterion descriptions we have selected 
with view to their validity or relevance. 

A variety of approaches may conceivably be applied to 
judging the validity of criterion estimates. Often the researcher 
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concerns himself only with "face" validity, where the method for 
estimating the criterion behavior is superficially judged to be re 
fleeting the criterion behavior it purports to measure.. The be- 
havior elicited is assumed to be isomorphic with the criterion 
behavior . 



Sometimes an approach which I will refer to as ” postula- 
tional validity” is employed. Here the method and estimates for 
assessing the criterion behavior are judged, in light of postulated 
relationship of the behavior elicited to the criterion behavior, to 
be measuring the criterion. Various sub-approaches to the determina- 
tion of the postulational validity include: validity by definition; 

validity judged from the existence of reliable differences between 
individuals when the method is applied; content validity, or assump- 
tion of validity based upon estimates derived from selected samples 
of the criterion behavior; and validity in terms of . conceptual con- 
sistency-validity of the estimation method judged in light of the 
apparent relationship between estimates provided by the method em- 
ployed and some inferentially identified « construct” or behavior. 

Further important considerations with regard. to criterion 
estimates or measurements have to do with the reliability of data 
yielded by a particular method; and also with the feasibility, or 
practicability, of an estimating technique. Certainly these cannot 
be neglected. 

Approaches to Criterion Definition and Description 

Returning to the matter of descriptions of criterion be- 
havior, I should like to simply note some of the approaches that 
may be employed. 

Criterion description is basically a function of thorough 
and detailed acquaintance with the behavior we are dealing with— in 
this case, the behavior of teachers as they carry out. the responsi- 
bilities demanded of them in particular school situations. Such 
acquaintance usually is best acquired by controlled observation. 

This is a particularly important consideration. Too often, . I sus- 
pect, we try to accomplish criterion description by arm-chair and 
associative recall methods. The generality and usefulness of a 
criterion description is likely to be proportionate to the extent 
that essential details of the behavior under study have been iden- 
tified and classified. And the most appropriate way of becoming 
knowledgeable about behaviors that may contribute to the criterion 
is by observation under controlled conditions. 

Generally speaking, the usefulness of satisfactoriness of 
a criterion description will be greater when: 
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(a) the criterion behavior under consideration, or its 
products, can be operationally described, directly ob- 
served, and objectively recorded; 

(b) the possibility of varied interpretations of. the 
criterion behavior and its products by different indi- 
viduals is minimum; 

(c) the observations directed at the identification of 
criterion behaviors and the data based on observations 
are analytical rather than global; 

(d) meaningful aggregates of the criterion behavior are 
distinguishable from irrelevant behaviors and attention 
is given to the determination of such behavior patterns; 

(e) the investigator is cognizant of, and attentive to, 
the more prevalent sources of criterion bias (e.g., 
contami na tion by concomitant behaviors, by ” opportunity,* 
by experience, by rating sets, etc.; deficiency , or 
incompleteness of the criterion; and criterion distortion); 

(f) observations directed at identification of criterion 
behaviors have been extensively replicated (e.g., an ade- 
quate number of individual cases have been observed and 
observations conducted in a variety of times, places, and 
circumstances ) . 

And we must remember that underlying all criterion descrip- 
tions is the matter of identifying the prevailing values. and expecta- 
tions that form the context for, and dictate,. the criteria we formu- 
late. We must first seek answers to value oriented questions such as 
Are teachers expected to be permissive with regard to pupil behavior? 
Are they expected to maintain rigorous standards of pupil learning 
and control? Are teachers expected to be rigid disciplinarians? Are 
teachers expected to be highly knowledgeable about subject matter 
content? Are they expected to be available repositories of subject 
matter knowledge, or are they expected to arrange for the. pupil to. 

» discover" information? Are they expected to take an active part in 
directing learning, or to arrange learning situations for individual 
progress? Are teachers expected to participate in administrative 
policies and decisions, or are such matters to be left to the 
administrative staff? 

Obviously these are only a few questions illustrative of a 
kind that might be asked in trying to assess the value climate of a 
community or school. The questions referred to admittedly relate to 
global sorts of values— behavioral descriptions would have to be de- 
rived in greater specificity to be useful. Questions of this sort 
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do not refer to all-or-none value judgments. They are not neces- 
sarily mutually exclusive. And the answers do not spell-out the 
behavioral criteria to be employed in judging the validity of a 
teacher selection policy or procedure. Nevertheless, such ques- 
tions, together with many others, do provide the necessary first 
step of determining the value climate before the process of desig- 
nating specific criteria can be engaged-in. 

The actual description and definition of criterion be- 
havior may follow a variety of approaches or strategies. All too 
frequently (or so it appears at least) no strategy at all is fol- 
lowed, i.e., criterion definition is completely neglected, or at 
best given only brief attention resulting in non-critical assump- 
tion of the criterion behavior involved. Among the studies I have 
reviewed I find practices covering a wide range of acceptability: 
completely non-critical assumption of the criterion behavior 
(either failure to consider criterion definition or unsophisticated 
acceptance of a criterion definition with no attention to its (a) 
completeness or (b) freedom from contaminating and distorting con- 
ditions); criterion description based upon analyses of judgments of 
presumably qualified authorities ; criterion description based upon . 
the analysis of responses to some response-evoking technique which 
is hypothesized to reflect some criterion behavior (here the cri- 
terion description is derived from the method of estimating the 
criterion — a procedure that should give us pause); and criterion 
behavior identified by analysis of records based upon observation 
of (a) behavior in situations presumed to involve the criterion be- 
havior or (b) products of behavior in situations presumed to in- 
volve the criterion behavior. 

Approaches to Obtaining Criterion Data 

Assuming we can describe our criteria satisfactorily we 
can now turn to ways of obtaining criterion estimates, i.e., the 
basic records and indices of criterion behavior against which data 
derived from selection procedures may be compared. 

As we have noted before a variety of methods of estimat- 
ing criterion behavior are available — methods which vary in ration- . 
ale and also in usefulness. May I just mention some of these in 
outline form: 

Some methods of obtaining criterion estimates 
A. Obtaining samples of the criterion behavior 

1. Direct measurement of samples of the criterion be - 
havior in process (i.e., on-going behavior) — 
primary criterion data. 
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a. "Natural" behavior — i.e., uncontrolled typical 
behavior 

b. Standard samples of the criterion behavior 

(l) Direct observation and assessment of behavior 
V (including interview) by trained observers 

(a) some observation approaches 

—Systematic, with immediate assessment 
(time sampling) 

--Retrospective (nonsystematic) 

—Analytical 

—Global 

— Relative 

—Absolute 



(b) procedures 

—Rating devices 
— Check lists 

(2) Observation and assessment of preserved 
records of criterion behavior in process 
(e.g., video tapes) 



(3) Assessment by untrained observers 

2 . Measurement of samples of products of the criterion be 
havior — presumed products of prir^y criterion data 

a Direct observation and assessment of samples of 

behavior products e.g., on-going pupil behavior 

—Uncontrolled products (i.e., products in natu- 
ral situations ) 

—Standard samples of products 

b. Use of devices for immediately eliciting the 

products of criterion behavior 

(1) estimations of maximum performance, e.g., 
pupil test results 

(2) estimation of typical performance, e.g., 
pupil responses to personal reaction ques- 
tionnaires (self reports of opinions, 
temperamental responses, etc.) 

(3) Measurement of (a) change in process, or 
(b) change in product 
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(a) change in estimates of samples of di- 
rectly observed on-going teacher 
behavior 

(b) change in estimates of samples of a pre- 
sumed product of the criterion behavior, 
i . e . , pupil behavior 

B. Identification of correlates of the criterion behavior 
(i.e., behavior "in process" or products which may be 
used as signs of the criterion behavior)— secondary 
criterion data. 

In my opinion the most valid of the various methods of es- 
timating criterion behavior is that of focusing upon samples of the 
on-going criterion behavior and resorting to direct estimation 
based on observation of these samples of criterion behavior in 
process . 

Ideally, in the study of the validity of teacher selec- 
tion procedures one would prefer to work with "identical elements" 
of the criterion behavior in which he is interested — to directly 
observe and directly measure the samples of the criterion behavior 
on which attention is focused. We would like to employ measure- 
ments based on "work samples" or the "natural" or "typical" be- 
havior in process, or, as a second best choice, upon similar ob- 
servations of a product of the criterion behavior. In many cases I 
think we can accomplish our study in this manner. In others, it is 
true, we must be satisfied with the indirect estimates or corre- 
lates of the criterion behavior against to which judge our teacher 
selection procedures. Such correlates-type estimates may involve 
(a) behavior or products from simulated situations (e.g., perform- 
ance situations, simulating those situations in which the criterion 
behavior occurs) or, (b) even presentation of graphic and/or verbal 
descriptions of situations involving the criterion behavior. 

As I come to the conclusion of my remarks I feel a strong 
sense of inadequacy; of having bitten off more than I can chew. As 
is the case with most of you present, I have given a great deal of 
thought to the problem of the criterion, particularly as it relates 
to teacher behavior and to the problem of validity study of teacher 
selection devices. I find it easy to identify and recognize many 
of the problems and difficulties with which we are faced in. trying 
to develop satisfactory descriptions of the criterion behavior of 
teachers and techniques which will yield valid estimates of the 
criterion behavior involved in teaching. I recognize the sources 
of bias in the description of criterion behavior and the conditions 
making for invalidity of the estimates yielded by different methods 
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of assessing criterion behavior. But I am admittedly frustrated by 
the difficulties involved in obtaining criterion data which are, on 
one hand, inclusive and complete and, on the other, exclusive and 
free of contamination . I know it is not easy to lick these prob- 
lems, particularly when we must frequently conduct validity studies 
in situations where we have been using certain teacher selection de- 
vices that were selected on a priori basis without the benefit of 
guidance of adequate criterion descriptions. And now, after the 
fact, we are faced with the problem of providing procedures that 
will yield estimates of criterion descriptions against which to 
test our selection data. I do not think the situation is an impos- 
sible one, but I cannot help but recognize, as I think most of us 
must, that we are faced with practical considerations which force 
us to compromise and employ make-shift methods that preclude the 
carrying out of validity studies of the quality we would like. 
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Some Notes on Validating Teacher Selection Procedures 

Donald M. Medley- 
Educational Testing Service 



As far as I know, all teacher selection procedures pres- 
ently used are based on the model of aptitude testing ; in other 
words, in constructing selection instruments the effort has been 
to devise a battery which would predict success on the job. Val- 
idation studies have, accordingly, sought to establish predictive 
validity against some kind of a criterion measure of teacher ef- 
fectiveness obtained after the teacher has been admitted to employ 
ment . 



A selection battery that could do this job fairly well 
would be useful indeed; it would enable the selection agency to 
compile a list of candidates with the candidate who would make the 
best teacher at the top and the one who would make the poorest at 
the bottom. Then it would be possible to appoint as many teachers 
as were needed in a given year, beginning at the top of the list, 
knowing that the best possible set of candidates had been chosen. 
This is a beautiful ideal; but it just will not work in practice. 

To ry mind, the sooner all attempts to validate selection proce- 
dures in this way are abandoned the better. 

In the first place, when you consider the nature of what 
you are trying to predict — teacher competence~it seems highly im- 
probable that it can ever be measured with a paper -and -pencil test, 
or any other device which could conceivably be used on the scale 
necessary for teacher selection in large cities. There is con- 
siderable experimental data which confirms this pessimistic point 
of view. Most of the predictive validities obtained in studies 
done in the past have been below .30; very few have exceeded .UO. 

And the improvement in predictive efficiency obtained with such 
small correlations is practically negligible. 

In the second place, even if the. correlations obtained 
were large enough to improve selection, their value would be sus- 
pect because of the limited validity of the criteria on which they 
would have to be based. At the present state of the art of measur- 
ing teacher competence, it is fair to say that the part of any 
teacher effectiveness criterion we can predict with a selection 
test is probably irrelevant to teacher competence anyhow. 

Finally, it is extremely difficult actually to carry out 
a validity study based on this model, since to do so requires that — 
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for experimental purposes — a good sized random sample of all who. 
apply for positions in a school system be admitted to teaching with- 
out any kind of selection. This is probably illegal, but in any 
case is not practical, in most large school systems. 

A more fundamental reason for abandoning the aptitude. test 
model is the fact that its use is based on an untenable assumption; 
the assumption that the major factors which determine whether a can- 
didate will succeed or fail as a teacher operate before he enters 
the school system. Such things as what kind of a. school and. neigh- 
borhood the teacher is assigned to, the characteristics of his 
pupils and the facilities and materials available to him when he 
tries to teach them, and the amount and kind of support he receives 
from his peers and superiors in the school system are seen as dis- 
tinctly less important than such things as what college courses he 
has had and what he learned in them, whether he has worked in. sum- 
mer camps during his undergraduate days, and how happy his childhood 
was, in determining his future as a teacher. This assumption is 
clearly implied by a conception of the selection problem. as one of 
identifying among applicants for positions those predestined to be- 
come competent teachers. 

It seems more realistic not to assume that the future of 
any of the candidates has been (or should have been) decided at the 
time when the selection takes place, but only that the candidates 
will vary in the degree to which they have mastered that part of 
their preparation which can be obtained before they enter the sys- 
tem. The selection problem would then be seen as one of assessing 
past learning rather than one of predicting future performance. 

The problem of validating a selection battery would then 
become a matter of content validity rather than of predictive val- 
idity; and this kind of validity is much more likely. to be achieved 
by a paper -and -pencil test or one of the other techniqv.es which a 
practical selection battery is likely to contain. From this point 
of view it may be said that teacher selection should be based on an 
achievement model rather than a prediction model. 

After a teacher has been admitted to probationary status 
in the school system, we are still faced with a problem of eliminat- 
ing those candidates who have satisfactorily completed their pre- 
service training but cannot teach successfully. This could be re- 
garded as a problem in prediction; but I prefer to say that it is 
another problem in achievement testing. The first years of a. 
teacher 1 s career should be viewed as a part of his training, if by 
the end of his probationary period he has not learned to teach, he 
is not ready to be admitted to permanent tenure and should probably 

be let go. 
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If proven ability to teach were made a prerequisite f 

the teaching of a standard lesson 

a^Xe valTsrectiin instruct than anything that 
could be used at the pre-service level. 

If we adopt the achievement model for validating selection 

improving the quality of teaching in the schools. 

There is one approach to this problem which (so far as I 
know) has never been tried out. This * teachers in the 

school S sS e r toanXogy to quality dSj^ts 

s*sS»f™ nrssu «. «. T 

but if objects so examined are random y s ’ lity of the 

obtain a quite satisfactory estimate of the average quail y 
entire output by inspecting only a sample of it. 

Tn similar fashion, the city school system could select a 
_ -i j? -j -| teachers who earn tenure in "the 

stratified random sample of all 1 ^d^areful study of each 
system each year, and make a ? The resu its of the study as 

teacher after he has — Tia~b e kept confidential, and guaran- 

tees* ^de° tSrt they'would no^affect 

on the basis of which changes could be made. 

The fact that only group scores would be used would make 

« -vv a tVi at 

are too low to oe useiux , , email relative to the total 

the number of teachers tested would be small relativ ^ ^ rela . 

number employed in the system w °“ Droce dures Although we do not 

«£, « — 
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in a system, using techniques already available to us, at a per 
teacher cost which would be quite reasonable. 

I have introduced this idea of quality control of teach- 
ers in a system in the context of the problem of monitoring teach- 
er selection policies and procedures, because that is our immediate 
concern here $ but I would point out the obvious fact that such in- 
formation would have many other uses, some of which might be re- 
garded as more important than this one. I see no reason why all of 
these purposes should not be achieved at the same time. Nor do I 
see any other feasible way of achieving the one we are concerned 
with — that of assessing the effects of selection policies and 
practices as they are used. 



Criterion Problems in Validating Teacher Selection 
Policies and Procedures 

Harold E. Mitzel 

The Pennsylvania State University 



Instead of following my assigned topic for this brief 
Daper with its emphasis on problems, and in order to stimulate dis- 
cussion, I have chosen to propose a research and development proce- 
dure which would lead to a new set of teacher selection procedures. 
By adopting this stance, I hope to convey the notion that many of 
our so-called teacher selection criterion problems are of our own 
making and generally stem from an inadequate conceptualization of 
the task. My proposed solution is, I'm sure, not without diffi- 
culty in implementation. Further, I'm well aware that the proposa 
is deceptively simple, fantastically expensive, an po 1 ay 
difficult to bring about. 

If we look at teacher selection as a data- gathering de- 
vice instead of as a procedure, and if we view the result as mul- 
tiple variables instead of as a single binary selecte -rejec 
scale, then it is clear that we have been looking m the wrong be- 
havior domain for the critically relevant information about teacher 
selection. Psychologically oriented disciplines have taught us 
that the best predictors of a personas future perforce are vari- 
ables based on'* his performance in a prior similar situation. Thus, 
we use a student’s high school academic record as a major trust- 
worthy indicator of his college success. . In industry, the best pre- 
dictor of performance on a specific job is the quality o a person s 
Derformance on previous closely related jobs. In the mill ary, 

pensive simulators and training aids have been used a ®. ^conditions 
of data for forecasting adequate performance under battle, conditions 

To be sure, the data derived from these sources are not perfect ^ 
forecasters of direct on-the-job behavior, but as a clas s 
ables they seem to stack up better than the currently use pre “ 
ToFTof teaching success. The real problem of teacher selection is 
to choose from among all the candidates who have met the same eli- 
gibility standards (i.e., baccalaureate degree, passed a set o 
state-approved certification requirements) the ones who will pro- 
vide the best pattern of teacher behaviors in the ^^m. Of 
course, what teacher behaviors are wanted have to be * d ® nbl ^ e * 
made explicit, but this task should be easily accomplished by the 
educational leadership in any given school system. 

After screening out eligible teaching candidates with ob- 
vious physical and psychiatric defects, I consider that symbol 
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manipulating ability as assessed by paper-pencil devices is the 
major variable which saturates the selection procedures of most 
large school systems. Systems that use undergraduate grade point 
averages as indicators of teacher quality are undoubtedly tapping 
the same academic variable. Large school systems that employ a 
written test, particularly the short answer variety, are banking 
on the same general factor, but attempting to measure it as re- 
liably as possible. Indeed, many of the dubiously valid interview 
rating techniques require the rater to form some sort of global 
assessment of mental ability. I am not advocating the selection of 
unintelligent teachers for our schools, but it does seem to me that 
we have exhausted bookish intelligence in its wide variety of forms 
as a viable predictor of teaching performance. I am suggesting 
that we turn to measures of classroom performance derived from sys- 
tematic classroom observation as a data source for improving the 
selection of teachers. I am reminded of an analogy recently in the 
news. For years the annual slaughter on the nation’s highways has 
been attributed to a class of variables associated with the per- 
sonal adequacy of the driver, but suddenly, in spite of the power- 
ful propaganda influence exerted by auto makers, it becomes evident 
that there is an important class of accident and injury related 
variables directly traceable to the mechanical condition of ve- 
hicles and their lack of safety engineering. For teacher selection, 
we need to discover a whole new set of variables. 

To exploit the potential of a new and different set of 
variables for teacher selection first requires a decision to quit 
trying to find valid predictors of teaching performance in school 
grades, interview rating scales, oral and written examinations and 
U. S. Office of Education guidelines. Second, we must attempt to 
develop classroom performance variables in the context of the class- 
room situation, thus giving ourselves an optimum chance of producing 
valid predictors. From the standpoint of the scientific approach, 
the procedure is straightforward and can be listed in eight steps : 

Step 1. Develop empirical descriptions of classroom per- 
formance by systematically recording samples of observable 
behavior for existing teaching staff members . These ob- 
jective records of teacher behavior may be process- 
oriented or content-oriented, and preferably both. It is 
extremely important that the arbitrary global evaluations 
of "good teacher" and "poor teacher" be avoided in any at- 
tempt to record teacher behavior objectively. The re- 
search on systematic observation reported by Withall, 
Flanders and Amidon, Smith and Meux, Mitzel and Medley, 
and Bellack would make good starting points. 

Step 2. Construct a comprehensive sample observation 
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instrument from the data obtained by Step 1. Here the 
judgment of educators who know the teaching process and 
the subject matter will have to be employed in order to 
select those behaviors which fit together to make up pat- 
terns deemed desirable by the school. For instance, if 
pupil participation in classroom activity planning is 
wanted in a school system, then it should be fairly easy 
to identify a cluster of specific teacher behaviors which 
clearly foster pupil participation. Conversely, a cluster 
of specific behaviors which inhibit pupil participation 
can also be identified. Experience shows rather con- 
clusively that these behaviors can be put together with 
unit scoring to form a reasonably reliable scale. 

Step 3. Administer the sample observation device to all 
teacher candidates in a naturalistic classroom setting 
(i.e., student teaching, internship). Of course, if some 
of the candidates are obviously unfit for reasons unre- 
lated to their classroom performance (i.e., infection 
with a communicable disease, defect in moral character), 
then these may be eliminated in advance. This step is. 
important since it enables the selection staff to obtain 
a wide range of behavior patterns. Research by Medley 
and Mitzel shows that it is necessary to obtain multiple 
samples of the observed behavior of a given teacher can- 
didate in order for a reliable pattern to emerge. 

Step b. Employ all candidates. This is perhaps the 
hardest decision to make, even in times of a teacher 
shortage, but it is a necessary step in order to validate 
a new instrument. 

Step Systematically observe and score the on-job be- 
Kavior of all teachers in the employed sample. The scor- 
ing should be done on a priori determined trial scales 
which were generated from the extensive observations of 
the behavior of in-house teachers. This step should be 
accomplished as soon as feasible after employment in 
order to minimize the self-selection effects of early 
dropouts . 

Step 6. Using the observation data generated in Step £ 
criterion and employing optimum weights for the 
several behavioral components, rank order the sample of 
new teachers and establish a trial demarcation between a 
satisfactory group and an unsatisfactory group. (The 
procedure itself does not demand that anyone be dismissed.) 
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Step 7. Analyze the pre-employment observation scale 
claia against the on- job observation criterion obtained 
from the same individuals. This step makes it possible 
to assess the predictive power of the pre- employment 
observation information. 

Step 8. Repeat the process beginning at Step 2 on a new 
sample of teacher candidates utilizing improved observa- 
tion techniques and behavior scales. 

Now the oversimplified eight-step process described above 
has a number of pitfalls. If the pre-employment teaching situation 
in which the predictive data are gathered has in it a lot of content- 
specific or situation-specific elements, then the predictors do not 
have a maximum opportunity to be closely related to the criterion. 

For example, if a candidate is observed during student teaching in a 
comfortable, middle-class, suburban school and then is measured on- 
the-job in a slum school, one would expect this difference in situa- 
tion to have some impact on his instructional behavior pattern. 
Similarly, if the predictive measures for a candidate were obtained 
from a series of mathematics lessons and the criterion data were 
generated for the same person on several art lessons, some of the 
observed differences might well be attributed to the change in 
subject matters. 

These problems of the specificity of schoolroom situation 
and content can probably be lessened by careful planning in large 
school systems. 

That the use of classroom observation as a source of se- 
lection data would be a revolutionary development is confirmed by 
the recent survey reported by Gilbert and others. This 19&6 report 
shows that, in almost sixty per cent of the large public school sys- 
tems, candidates are not at all observed. In an additional twenty 
per cent, only one observation is made per candidate and the data 
derived from this procedure are undoubtedly a mass of subjective 
non-predicting rating scales which tell more about the rater than 
about the candidate. It seems to this observer that we must strike 
out in new directions on the task of predicting teacher performance 
and discontinue our perseveration with unproductive sources of data. 
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Needed Research in Teacher Selection 

John C. Flanagan 

American Institutes for Research 
in the Behavioral Sciences 



This discussion of needed research in teacher selection 
will discuss research needs under four main headings. These are 
m the determination of the pattern of teacher behaviors which de- 
« La effect!^ teaching, (2) the evaluation of the effectiveness 
ff teachinr P erfor^ce!’(3) ; the prediction of teacher effectiveness, 
and {h) factors which cause good teachers to leave teaching. 

It is proposed that research on determining the ^aviors 

which constitute effective teaching be given a new s °S!f instead of 
principal characteristic of this new orientation is ^atinstea 
a single set of specifications for the requirements of an effective 
teacher there should be a very large number of sets of requirements 
defining the behaviors which make for effective performance of eac 
o! the various aspects of the teaching task. Although such an ana- 
Stical Sproach has advantages under any circumstances it becomes 
of great importance in the situation where the activities of the 
teacher are likely to change markedly with the advent of ne 
in^methods , new objectives, new technology and the provision of 
clerical assistance by computers and teaching aides. 

Effective teaching in the past has involved many activi- 
ties, such as lecturing, maintaining discipline : ^ding dis cus- 
si-ons checking on students 1 learning, correcting students errors, 
stimulating students' thinking, and arousing students -interests. 
Although manv of these activities are likely to be impor a 
tffectlve^ching in the schools of the future, it seems probable 
that there will be considerable change in. emphasis.- More import- 
ance will be given such activities as individual tutoring; assi 
Sg students to pU their educational program; developing a sense 
o? responsibility in the student for his own educational develop- 
ment- helping him to discover his interests and special abilities, 
helDinghim^discover the satisfactions that can be obtained from 
creative acUvitils; and assisting him generally in his personal 
and social development. 

Since it is unlikely that most of us will be able to pre- 
riict the precise nature of the teacher's role and activities m the 
u i *p fn+urp it is of considerable importance that we 

carry out research to define the requirements for effective teacni g 
n-P manv kinds It is also important that in gathering this informa 
tiorTthe^characteristics of the individual student involved in the 
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teaching activity be clearly specified. It seems likely that the 
requirements for motivating one type of student are likely to be 
different from those for motivating another, for example. 

In carrying out research in this field, it is proposed 
that task analysis procedures and the critical incident technique 
be used. These can be supplemented by other observational and 
analysis procedures. A device which has already shown its value in 
this type of research is the video-camera which can capture the 
teaching situation and play it back for observation and analysis as 
many times as is required. The procedures of task analysis are 
likely to be of special value in developing requirements for new 
types of teacher activities. Of course in using this procedure, 
the characteristics of the student, his previous knowledge and many 
other factors can be specified and varied at will. The critical 
incident technique also makes possible the detailed description of 
the specific student involved and the definition of the particular 
aim of the teacher’s activities. It should be of great value to 
collect many thousand incidents each of which is classified accord- 
ing to the type of student and the specific teacher activity 
involved . 



The second area of needed research is in the evaluation 
of the effectiveness of a teacher. Of course in this case also, 
the focus should be on the effectiveness of the teacher in each of 
a wide variety of activities with each of a number of types of 
students. Teacher effectiveness may be evaluated in terms of either 
the process or the product. As assessment procedures are developed 
which measure all of the desired changes in student behavior, and 
computers provide efficient storage and analytical procedures for 
these data, it seems likely that much more detailed records can be 
collected and analyzed than has been characteristic of past evalu- 
ation procedures. It appears that if individualized educational 
development procedures are used which set specific obtainable tasks 
for each student, the details of whether a particular teacher’s 
students obtain the goals specified will provide a much more pre- 
cise record of teacher effectiveness than has been previously 
available. High priority should be given to research and develop- 
ment work on procedures for a comprehensive evaluation of the 
teacher’s effectiveness in many activities with many types of 
students based on systematic measurement of the effectiveness of 
the instruction. 

It also seems desirable to continue research on process 
variables in teaching. It is suggested that the most promising 
lines of research on process variables involve systematically de- 
veloped observational record forms on which specific observations 
by a competent judge are made of effective and ineffective perform- 
ance on the part of a teacher. These, of course, should -be related 



to the particular activity and the particular type of student 
involved . 



A factor which could have considerable influence on the 
quality of teachers is the use of merit increases and the assign- 
ment of tenure status only as a result of demonstrated effective- 
ness. The lock-step promotions characteristic of many school sys- 
tems and the failure to eliminate the ineffective teachers produce 
a situation attractive only to the mediocre and marginal teachers . 

Of course, improvement in these areas must be based on valid pro- 
cedures for the evaluation of teachers . Research on the evaluation 
of teacher effectiveness should be given much attention for another 
reason, which is its essential importance to other types of research 
on teacher selection. 

The third area of research relates to the development of 
predictors of effective teaching. It appears that long-range 
follow-up studies are among the most promising approach to this 
problem. It is anticipated that data from students tested as 12th 
graders in Project TALENT who have graduated from college and en- 
tered teaching will become available this fall. Similarly, in the 
next three years data from other classes will be collected and ana- 
lyzed. These data should be of distinct value in defining the type 
of individual who enters a teaching career. Of even greater value 
will be the ten year follow-ups planned for Project TALENT which 
will begin in 1970. These studies will provide data on those who 
have not only selected careers in teaching but also have been found 
to be effective teachers over a substantial period of service. As 
in the case of the other types of research studies mentioned, it 
seems important that predictors not be focused on the quality of 
teachers in general but rather the effectiveness of the teacher in 
performing specific roles or activities. It is also believed that 
the trend in research with respect to predictors will be in the 
direction of a very large number of specific predictors rather than 
a few more general measures. 

The last area of needed research relates to the condi- 
tions of teaching with special emphasis on those which have nega- 
tive selective value. It is well known that the turn-over in teach- 
ing is very high, and it seems important to determine why teaching 
loses some of its most effective individuals. One of the problems 
calling for research in this area is the effects of family duties 
on remaining in teaching. It would seem worthwhile to study the 
question of whether part-time employment or somewhat less than full- 
time employment might retain some of the better women teachers. 

Other research topics in this area relate to the study of specific 
teacher activities, particularly with the view to determining 
whether or not some of the clerical and administrative duties could 
be performed by teacher aides, clerks, or computers. 
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Another valuable study might be directed at the question 
of "why many promising young teachers leave teaching early in their 
careers." It seems likely that the administrative practice of as- 
signing new teachers to difficult classes, loading them down with 
the more unpleasant extra-curricular assignments, and similar pro- 
cedures which give the new teacher a bad initial impression of 
teaching are responsible for losing effective teachers. 



In conclusion it is believed that the new methods of 
teaching will make it possible to evaluate teaching effectiveness 
much more precisely than has been the case in recent years. This 
will make research on the characteristics for effective performance 
in specific teaching tasks both feasible and important. As pro- 
posals for new educational programs are developed they should be . 
accompanied by plans for research on the teacher *s role and activi- 
ties in the new program. Perhaps one of the greatest needs for re- 
search in teaching is the application of systems concepts to the 
teaching and learning system. This application would include com- 
prehensive measures of the input and output of the system as well 
as studies of the process variables. For such studies to be effec- 
tive much better measures of the input and output of the system in 
terms of assessing the students 1 behavior, concepts, and abilities 
will be required than are now available.. Perhaps the first step in 
a program of teacher selection should be the definition a eve op 
ment of measures of the products of teaching. Unless w ® haV ® Valld 
measures of the effectiveness of a teacher* s efforts, other types 
of research must be regarded as tentative, exploratory, and perhaps 
in many cases, trivial. 



Needed Research in the Area of Teacher Selection 

Harry B. Gilbert 

The Pennsylvania State University 



I Assumptions 

There are three basic assumptions that underlie the spe- 
cific proposals I shall outline. Permit me to make them explicit. 

1 . Interest in the area of teacher selection is minimal 
based u pon the actual amount of research under way\ 

Howeve r a great deal of interest does exist am dng teacher 
personnel selectors and universities. The pro blem is to 
make patent what’s latent. 

2 Profes sional teacher selection practice s are rarely enn 
nloved. In large school systems that presume to be using 
select ion techniques, screening is actu ally what is done_._ 
I n smaller, affluent school districts, hun ch rejections 
and global perusals, sometimes in actual ob servations^ 
serve as selection techniques . 

A moment’s reflection on the current shortage of 
teachers, particularly in those school districts per- 
ceived to be "tough”, will confirm the probability that 
this assumption is warranted . 

3 . Since th e field of teacher selection is a great big area 
up foi grabs, it is desirable that research be en couraged 
in" varieties Jf~"appro aches , without too much specificity » 
It fo llows, of course, that wide dissemination of re- 
se arch be encouraged and that investigators b e supported 
with the notion that hypotheses may be r ejected as well 
as V erified by experimental data. Regrettably this 
simpl e dictum, readily understood in univers ity setting_s^ 
seems to be a heretic notion in an age when i nnovations 
are publicized as successes before evaluation. 



II Proposals 

1. Attention must be paid to the process of stimulating at- 
tention to the methodology of teacher selection and to research, 
which must inevitably be an assessment of the validity of presen y 
employed procedures and those yet to be tried or designed. Surely 
we can agree that fundamental research on teacher behavior - and on 
pupil-teacher interaction as an outcome of the teacher-stimulus, 
must be encouraged. What is needed is understanding of why so im- 
portant a field for research continues to be neglected, despite 
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promising starts by such workers as Withall, Mitzel and Ryans. The 
simple answer - too difficult - seems not enough in an age when com- 
plex problems are tackled with the resources made available by con- 
temporary cash flow. 

I suggest therefore that social psychologists and sociolo- 
gists be invited to investigate broadly the field of research in 
teacher selection for their aid in understanding avoidance attitudes 
of potential research workers. There is no disposition to hide the 
obvious - that I have made a value judgment and that the purpose 
would be to be able to persuade research talent into action in this 
sphere. 



I propose too that we evaluate the effectiveness of the 
design of this conference, with its influential participants, as a 
means of stirring interest and action in teacher selection practice 
and research. 

2 . We should take advantage of current supply and demand in 
the area of teacher selection. Specifically I refer to the shortage 
of teachers as the country begins to wake up to the great need for 
teachers and pumps green blood into local systems from the great big 
Federal artery. Teacher shortage gives the personnel research man a 
rare opportunity. He can lower his selection cut-off scores to pro- 
vide a greater range for assessment of predictive validity. It will 
take some degree of courage to do this, although some reflection on 
the lack of predictive validity data should be encouragement enough. 

At the same time, there is always an over- supply of appli- 
cants for supervisory or administrative positions in education. 

This should serve to be a source of comfort for those who would ar- 
gue that one cannot be selective, or try to be, with teachers when 
we need all who apply. Furthermore, it is palatable to applicants 
to be fed a variety of selection techniques when there are more ap- 
plicants than positions to be filled. This presents the oppor- 
tunity to employ varied and multiple devices, to a socially accept- 
ing, even willing, group. 

3. I new come to some specific proposals. First, I make 
reference to the l6 " Suggestions for Further Research" on pages 
£7 of "Teacher Selection Policies and Procedures in Large Public 
School Systems in the United States." There should be enough ideas 
here to generate several bushels of Ph.D.'s or Ed.D.'s. 

I should like to take the time to express myself on the 
perennial dissuader - the lack of sound criteria. The fact is that 
so .nd criteria exist only in relatively simple occupations and not 
in complex professions. Worker output, in terms of quantity, or 
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quality, or time, or a combination thereof, can be employed as cri- 
teria for production line employees. But where do you go when 
everybody knows how hard it is to define "success in teaching" par- 
ticularly since this amorphous generalization (bad to begin with) 
keeps shifting in different school settings, at different levels 
and with different subjects? 

At least we have learned from previous study that the 
overall, general estimate of "success in teaching" is a concept to 
be discarded. We can work with dimensions of teacher performance, 
as the local hiring school system defines them. These dimensions 
may be with respect to teacher behavior, such as "evocative of 
pupil participation," "encouraging pupil-pupil interaction," "ac- 
cepting of deviant behavior," etc.. They may be with respect to the 
ultimate in expectancy, namely pupil development, m skills, 
knowledge, or attitudes. 

Let’s not beat a dead horse. We can all agree that re- 
search is necessary to refine and define the dimensions and to de- 
termine methods objectively - or reasonably objectively - to assess 
them. I propose that we accept crude estimates of the extremes m 
dimensions and not wait for the millennium. I propose that we en- 
gage task forces of teachers, teacher educators, school. adminis- 
trators and personnel experts to agree on working definitions o 
dimensions of teacher behavior on a scale from "most desirable to 
"least desirable." Then obtain nominations for extremes on these 
scales - the "best" and the "worse." Various procedures can be em- 
ployed for these nominations. They could be peer . nominative tech- 
niques, observer ratings, supervisor ratings, pupil nominations and 
measured pupil performance. Hopefully a variety of techniques 
would be employed. 

I am arguing in short, for our getting started with best 
available techniques to obtain criteria while others among us con- 
tinue more basic research. 

In another setting (APA Annual Meeting, September 19&6) I 
have also urged that we utilize the best informed personnel minds 
to review present selection procedures. While I would hardly clas- 
sify such activity as research, I do maintain. that some improvement 
in present procedures can ensue, without waiting for results of 
years of study. 

In short, I am urging a redirection of energy to the 
problem of teacher selection research and procedure, in a reasonable 
and practical way without sacrificing the ongoing need for basic 

research. 

Finally I would like to propose the need to establish a 
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clearinghouse xor research and practice in teacher selection. I 
don*t know who woulu establish this. I believe the field is im- 
portant enough that some university interested in developing this 
branch of educational administration might be encouraged to assume 
such leadership. Certainly a repository of findings with dissemina- 
tion facilities would be a major resource in development of interest 
in the field. 



SECTION III. 



CONPERENCE PROCEEDINGS 



Greetings 
Paul Denn 

Board of Examiners 



Good evening, ladies and gentlemen. Everything that we 
say from now on will be taped. We do have to telescope this eve- 
ning a good deal of what we have to say and therefore some of the 
humorous remarks which I attempted to gather together at the very 
last moment will have to be omitted. In connection with this tap- 
ing, please speak clearly and distinctly. If you ask a question, 
would you state your name so that it may be recorded. 

We are all seriously engaged in what many of us - I think 
til of us perhaps - think is just about the most important thing in 
education, the selection of good teachers for children of our 
schools. Whatever we can contribute at this conference towards the 
improvement of such selection will put us all that much ahead and 
the nation that much more in debt to all of us. 

I'd like to express the thanks of our board to the people 
who worked so hard on this conference: our chairman. Dr. Bogen, 
who I'm now informed is in the hospital and will undergo an opera- 
tion tomorrow; Dr. Gilbert, who worked very closely with him on the 
research committee; Dr. Lang and Dr. Kalick, both of whom were for 
a brief time our entire research staff. This evening I'm going to 
call first on Dr. Gilbert to outline for you the background and 
objectives of the conference. 

Background and Objectives of Conference 
Harry B. Gilbert 
Pennsylvania State University 



Thank you, Paul. I am here to outline the background of 
this conference and I shall be brief. I think it has to start with 
the role of the Board of Examiners in teacher selection. It's just 
a plain fact of the size of New York that New York City's Board of 
Examiners is the largest teacher selection agency in the country. 

It is involved more with selecting teachers than any other agency. 
Over 50,000 applicants a year, a school system that has 55,000 pro- 
fessional personnel at last count and still rising, a budget for 
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selection that's well over a million and a half, all of these facts 
give evidence of the large scale involvement in teacher selection. 
For years the Board of Examiners has been interested in improving 
its selection techniques, but the Board of Education has not seen 
fit to support research in teacher selection. The Board of Exam- 
iners employs one research associate and has not even been able to 
obtain funds for a secretary for him. This seems incredible and as 
I sit in my own little lair on Mt. Nittany now I find it hard to 
believe, but year after year after year, the Board of Education has 
simply refused to recognize that you can't spend all this money on 
selection without evaluating the process. At the same time we've 
grown tired of hearing criticisms by informed people that we ought 
to be doing some research on our selection techniques. The less in- 
formed people simply say " Abolish the Board” without even thinking, 
without even realizing the implications. But these important facts 
of our background, the desire to evaluate the process of teacher se- 
lection and the failure to get the support that's necessary has 
caused the Board of Examiners to look elsewhere for support in this 
very vital process. And in the looking elsewhere it has recognized 
that there is a high degree of interest in the process. The in- 
terest is latent and this is something we'll talk about later in 
the meeting, but all of this background is necessary to understand 
why the Board of Examiners has come up with a 5-step procedure to 
generate more interest in the process of teacher selection. I want 
to outline the five steps and then delineate the objectives of this 
conference. 

1. The first is a survey of procedures. The Board of Exam- 
iners undertook with aid of the US Office of Education to find 
out what large school systems throughout the country say they 
are doing in teacher selection. 

2 . Second, is this present phase - an invitational conference 
in which a group of experts representing industry, univer- 
sities, school systems, state education departments, civil 
service and teacher organizations, people who know the process 
of selection and can apply it to school systems, have been in- 
vited to come here to discuss the implications of the survey 
and to make recommendations for action. 

3. Third, we intend to disseminate widely the results ana 
recommendations of this conference. 

k- Four is the action step where we hope evaluation of pro- 
cedures, development of new procedures and research in the 
basic problems confronting those interested in teacher selec- 
tion will take place in a variety of settings. 
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And fifth, the establishment of a clearinghouse somewhere 
where all the results of research and the variety of forms and 
procedures utilized in teacher selection will be collected and 
then further disseminated. 

I now conclude with a simple and straightforward delinea- 
tion of the objectives of this conference. There are three objec- 
tives as I see them. In the first place we have just completed a 
survey of large school systems, the largest and some that are 
reasonably large. What does this mean in terms of selection? 

There are some obvious implications of the survey that personnel 
people with experience and know-how can make with respect to on- 
going procedures. More important though, what kind of research do 
we need to do? Secondly, having looked at the implications of these 
surveys, let’s put them in very specific terms with respect to 
teacher selection practices . This is not to say that we forty 
people here can recommend action which we know will be followed . 

We do think, however, we should set forth what should be done in 
terms of needed research in teacher selection, and any related mat- 
ters that may come up. You just can' o lay out ideas to a creative 
bunch like this expecting that it will follow a pattern. And this 
in turn should result in a summarizing document which can then 
serve as a guide for research and practice throughout the country. 

We think - and I'm quite frank and bold about what I'm about to 
say - we think that the prestige and the brain power nestled right 
here which might result in a document which says this is a set of 
useful guides for people who are interested in teacher selection 
might then serve as an important stimulant for the country. We 
hope it moves in directions that can't be predicted right now. But 
above all, we hope it moves because right now the whole field of 
teacher research and teacher selection is unfortunately stagnant 
and very vitally important. Thank you. 

Paul Denn: I'd like now to introduce Miss Gertrude Unser, 

who was chairman of our Board last year and who will lead tonight's 
panel discussion. 



Session No. 1: 



Teacher Selection Methods in the United States 

Chairman: Gertrude E„ Unser, Board of Examiners 

Teacher Selection Methods in Large Public School Systems 
in the U.S.: Gerhard Lang, Montclair State College 

Teacher Selection Methods in 62 School Systems Affiliated 
with the Metropolitan School Study Council: Perry M. 

Kalick, Hunter College of the City University of New York 

Gertrude Unser: You know, I'm an old school teacher and 

I think one never really overcomes the habits of a lifetime. Every 
school teacher assumes that the class has done the homework but, 
just in case, she then tells them what they should have learned. 

I'm assuming that you have read the two publications, one called 
"Teacher Selection Policies and Procedures in Large Public School 
Systems" and the other called "Teacher Selection Policies and Pro- 
cedures in School Systems Affiliated with the Metropolitan School 
Study Council." The first was the major piece of research under- 
taken and dealt with public school systems of a certain size, those 
having 12,000 or more pupils. The other was a parallel piece of 
research related to and growing out of the first, but dealing with 
smaller school systems that were relatively more favored economi- 
cally. I think that each of us brings to the reading or the study 
of this kind of document a certain predisposition, but if you have 
read them carefully, I believe that certain conclusions are quite 
inescapable. Despite the fact that there has been considerable ex- 
perimentation in the curriculum, methodology, and organization of 
schools in recent years, we're struck in reading these documents 
(and we know from our own experience too) by the fact that in the 
procedures used to select teachers we're still operating in a sort 
of horse-and-buggy age. The conclusion is inescapable that many of 
the techniques used are really dictated more by expediency, that is, 
the availability of applicants at a particular time or considera- 
tions of cost in time and money, rather than reasoned and knowl- 
edgeable considerations of what are the best selection procedures . 

I think, too, that we all know that there is a dearth of research 
in the validation of selection procedures against other criteria. 

I am particularly interested, and I think other members of my Board 
are, in understanding the effect of the pressures in a period of 
teacher shortage upon selection procedures. There are many more 
things that strike one as one reads these documents . I don t want 
to go into them now because I assume that you have read them and I 
assume also that the following speakers will talk about them, but 
just to refresh your memory concerning the procedures and conclu- 
sions of these studies, we are going to call upon Dr. Lang* who was 
a research associate with the Board of Examiners when this study was 
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being made, to discuss very briefly the teacher selection procedures 
in large public school systems. Dr. Lang is now an Associate Profes- 
sor of Education at Montclair State College. We *11 then call upon 
Dr Kalick to discuss the parallel study, the selection procedures 
used in smaller school systems that are more favored economically. 
There are certain striking resemblances and certain differences. 

Their presentations will be brief and then I think they will be 
happy to answer any questions which you may have. Now we *11 ask Dr. 
Lang first to discuss the major study, the procedures used a.n large 
public school systems. 



Gerhard Lang: I see that Miss Unser operates under the 

same illusion that we do in the colleges, namely, that people do 
their homework. We hope, however, that most of you have had a 
chance to browse through the report on our study of large public 
school systems in the United States 1 or have at least digested the 
summary of this study. My purpose here is not to recapitulate the 
mass of statistics, but rather to refresh your memory, to point out 
a few salient features, and to answer your questions regarding the 
various aspects of this study, which is the point of departure for 

our conference . 



We conducted a survey of 320 large public school systems, 
defined as those systems which have at least 12,000 pupij-s, with 
respect to 1? areas of teacher selection policies and procedures, 

namely: 

1. Analysis of existing staff resources 

2. Preparation and use of job descriptions 

3 . * Resources used in the recruitment of applicants . 

1;. Means of giving prospective candidates information re- 
garding the school system to which they are applying 
£. Non-local selection of teachers 

6. Use of the application form 

7. Professional preparation required for teaching positions 

8. Use and follow-up references 

9. Use of examinations (written, oral, physical, etc.) 

10. Interview techniques in teacher selection 

11. Classroom observation of candidates 

1 Harry B. Gilbert, Isidore Bogen, Gerhard Lang, and Perry M. Kalick. 
Teacher Selection Policies and Procedures in Large Public School S^re- 
tems in the United State” "Unpublished report, Cooperative Research 
pj^ject n£7 S-33U7 Office of Education, U.S. Department of Health, 
Education, and Welfare. 1966. 63 p. 



12. Use of eligibility lists 

13. Timing of notification of appointment 

Ik. Declination of offer of appointment 

l£. Appeals from the decision of the selecting authority. 

What were some of the major findings? We found that only 
about 2£ per cent of the respondents prepare job descriptions for 
teaching vacancies and only £ per cent use a specific job descrip- 
tion form. Less than 13 per cent of the school systems give exam- 
inations as part of their selection process • Physical examinations 
are required by .about 60 per cent of the school systems, or con- 
versely, hO per cent do not require a physical examination, indeed 
a most surprising finding. All large school systems interview can- 
didates. About 30 per cent of the systems almost never or only oc- 
casionally train their interviewers. While the time allotted to. 
the interview ranges from 10 minutes to over one hour, the majority 
of school systems (££ per cent) devote between 20 to 30 minutes to 
their interview. In this short time interval, interviewers are ex- 
pected to assess up to 13 characteristics of the candidates, e.g., 
"ability in the subject matter that the candidate proposes to teach," 
"logical thinking," "attitudes towards his work," "potentiality for 
professional growth," "personal appearance," and "philosophy of. edu- 
cation." One may well question the validity as well as the. relia- 
bility of the interview conducted by the typical large public 
school system. References are almost universally requested from 
former employers and college professors. Surprisingly, however, 2£ 
per cent of the school systems ask for testimonials from friend^ 
of the candidate, a rather questionable procedure. What about sam- 
pling the candidate’s teaching ability? About 60 per cent of the 
school systems do not observe candidates, only 20 per cent of the 
systems tend to make one observation, and 6 per cent apparently 
make two observations. Despite our chronic concern about attract- 
ing people to enter a teaching career and our considerable invest- 
ment in the selection process, large school systems tend to notify 
candidates rather late of their appointment. May seems to be the 
modal month of notification. Moreover, the larger the school sys- 
tem, the greater the tendency to notify candidates later . One can 
only conjecture as to how many candidates are lost because of the 
lateness of their notification. 

These and the many other findings of our study are sub- 
ject, of course, to a number of limitations. We have relied on a 
detailed questionnaire as our data- gathering instrument and have, 
assumed that the respondents gave us honest and correct information. 
We must also bear in mind that the conclusions drawn from the data 
are pertinent only to the population of 320 large public school 
systems which had returned useable questionnaires. The findings 
cannot be generalized to populations of other school systems. 



Nonetheless, the analysis of our data left us with a. 
strong impression, reinforced by our visits to representative in- 
dustrial concerns, such as American Telephone and Telegraph, 
Standard Oil of New Jersey, Metropolitan Life Insurance, Macy s. 

New Jersey Bell Telephone, and the Port Authority of New York. The 
impression is that, compared to generally accepted practices used 
in industry to select personnel who are at a professional level 
similar to that of teachers, teacher selection practices in large 
public school systems appear to be by and large inadequate and un- 
sophisticated. It seems that more and better efforts are expended 
to select first or second level supervisors than teachers for our 
large public school systems. 

Realizing that there exist many gaps in our knowledge of 
what constitutes "effective teacher selection practices," we have 
concluded our report with a series of l6 suggestions for further, 
research. Our survey represents only a beginning. We are. optimis- 
tic that this conference and its brain-storming sessions will bring 
us closer to our goal - to select qualified teachers, so vitally 
necessary in this critical period of American society. Thank you. 



Perry Kalick: The questionnaire that Dr. Lang referred 

to was also used in a study of a group of school systems in the 
metropolitan area. These systems are the Metropolitan School Study 
Council Systems (MSSC). They* re clustered around New York City and 
they've engaged Teachers College, Columbia University, to do re- 
search for them. Two of the school systems were in Connecticut, dd 
in New Jersey, and 38 in New York. What are the characteristics ° f 
these school systems? On an average they spend about two-thirds, 
more per pupil than the large public school systems (LPSS). Tneir 
median size is approximately l£0 teachers and they have abouo 51 
teachers per thousand pupils as opposed to 38 teachers per thousand 

pupils for the LPSS. 

What are the main findings of this study, which compared 
teacher selection practices in the MSSC systems with those in the 
LPSS? As is the case with the LPSS, the MSSC systems use the inter- 
view as the prime selection device. The MSSC systems make a 
greater effort than the LPSS in extending the radius of search for 
candidates; they make twice as great an effort as the LISS in the 
search for candidates beyond a 500 mile radius and somewhat more 
than twice the effort beyond a 1000 mile radius. MSSC systems exert 
proportionately three and one-half times the. effort to make at least 
one classroom observation of a candidate. Ninety- two per cent of 
the MSSC systems telephone the recommender of a candidate as con- 
trasted with 66 per cent of the LPSS. The MSSC systems attempt to 






117 



*s2. i sr5“.s^r.s »££ 

sras fiiSSHSd - . *™. rV f *. =. — 

pffpctive than one individual making the decision. As compared wito 
+Vip T PSS proportionately almost twice as many MSSC systems will - 
f^o^the” committee approach in the selection of ^ates^ Vtoe^ 
L/uqqp ovstems do employ a committee, they will involve the pnn 
Cioal fouf times as frequently and the superintendent of schools 
five times as readily as the LFSS Of course thxs xs^ ^ ^ 

JSKo'p? notify candidates of their 
selection by the end of April. 

It's not all one-sided, however. The smaller school 
systems are less likely to have interviewers who are traxnedxnt 

SS'^SiS: 

orocessing systems in the selection process. What are the xm- 
-i . „ .. here? Very likely, the advantages inherent xn smallnes 
0/ size coupled with the comparatively high expenditure level per 
“f, the factors which account for the superxorxty of MSSC 
sXol^vsteL over ?he large public school systems with respect to 
thf^selectiorT practices that I have cited. A small system wxth a 

high expenditure level per pupil might be xn a ^^ P °hf“le!tion 
nrovide a greater proportion of administrative time to the selec^io 

£5S5 'SHS **sssr 

which P are clustered around other very large school systems. 

Discussion 

rpr-tr-ude Unser: Now if you have some questions that you'd 

like to address to litter speaker, I think they'd be happy to try to 
^er them ?he questions may be based not only on thexr brxef 
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presentation, but on your study of the documents to which they re- 
ferred. They needn't necessarily be questions; perhaps you have 
comments that you'd like to make. 

Charles Mackey: I'd be very interested in having Dr. 

Kalick elaborate further on the use of data processing on teacher 
selection device. 

Perry Kalick: By the use of data processing I mean to be 

able to develop cards, program cards which list characteristics, 
qualities of the staff that you currently employ as well as the need 
in various vacancies as they exist throughout the system. The match- 
ing of these is made easier by having data processing particularly 
in an exceedingly large system like Chicago or New York City. The 
data processing also allows for articulation; with job descriptions, 
needs can be tied in with characteristics that have been established 
for each candidate or teacher. 

Charles Mackey: Has this been used successfully so far? 

The reason I ask that is not to be facetious, but that the frequency 
of divorce among sponsors who have been matched according to data 
processing reveals a higher rate of divorce than in any other type 
of matching. I was wondering if this would apply to teacher selec- 
tion as well. 

Perry Kalick: I'd like to think that the emotional fac- 

tors here are not involved, although the emotional factors in work- 
ing in a specific type of classroom are very much involved. And 
this is most important - that we identify the characteristics of the 
vacancy with the characteristics of the teacher if we know the 
characteristics of the teacher. 

Charles Mackey: Is this process being used now? Data 

processing in teacher selection? 

Perry Kalick: Yes, in only £ per cent of the MSSC school 

systems and some 29 per cent of the large public school systems. 

Charles Cogen: I guess this is a question that applies 

to this entire conference not only what we have heard tonight. And 
perhaps I shouldn't ask it, but maybe the answer is obvious in the 
light of what Harry Gilbert said in the beginning . Has there been 
any attempt at all anywhere to make any study of the outcomes of 
the examination system - that is, the product, any correlation be- 
tween the competency of the teachers as shown on the job and any of 
the processes used in the examination system? Do you have any such 
information? 

Perry Kalick: Charles, I think if we had the answer to 

that we wouldn't be here. The fact is that there are very few 
studies of the predictive validity of any selection devices in the 
teaching field. It's our hope that we will stimulate this and 
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we are going to develop this point at much greater length in the 
next day or two so I don’t want to belabor it. Let me just point 
out that we’ve made a few little passes at this and we have some 
tentative findings - it’s not our purpose to go into those now - 
that lead us to suspect that we have some promising paths to pursue 5 
but some of the basic research on it, even reliably assessing what 
happens in the classroom, are still to be done. They have been 
opened up and started . 

Harry Rivlin: I'm wondering whether there isn't a basic 

fallacy underlying the studies and the procedures of this and every 
other teacher selection agency. There is a sort of implication 
that one knows what a good teacher is, and if one knows what a good 
teacher is, then one can test to see whether a person is or isn't a 
good teacher. Various attempts to define what a good teacher is 
have failed - some more expensively, some less expensively. Instead 
of selecting good teachers, the school's job is to develop them. 

The selection of teachers is based better not on what happens before 
a person begins teaching, but after. As the various criteria were 
read off, the one thing that I don't think I heard was an evaluation 
of past performance. There is another kind of agency that selects 
teachers, our universities. When our universities select their 
faculty, the one thing that they do most intensively, if it's an in- 
stitution that cares at all about selection, is to make a thorough 
study of the past performance of the candidate. They don't check 
his eligibility because there are a great number of courses being 
taught in college which weren't available when the members of the. 
faculty got their doctorates. It isn't, "Has he had the courses in 
this area?" but, "Can he learn them?" Most of the university's 
best effort goes into the evaluation of past performance. In the 
case of beginners in college teaching, they can't evaluate past per- 
formance, but the universities will evaluate, and evaluate rather 
rigorously the performance before a person is granted tenure. In 
short, you have the conflict between the need to be legal, the need 
to be impartial, and the need to be intelligent about how you find 
the best person for the job. When the need for new teachers is 
great, you're not really selecting, you're rejecting those who are 
obviously unsatisfactory for one thing or another. Does it make 
much sense to spend so much effort in determining whether this fel- 
low gets an 82 or an 8l - it's really a pass or fail. In short, I 
wonder whether the emphasis cannot be put first on judging a per- 
son's potential by what he demonstrates, making the major part of 
the examination not the pre-service part but the post-service part 
or the in-service part, and in the case of people with experience, 
developing ways of evaluating what they have done rather than try- 
ing to disregard all that, or at least placing very little emphasis 

on it. 

Perry Kalicks I wanted to make a short comment — I think 
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this is why we're here. We're going to study the implications of 
this. As Dean Rivlin says, if we don't know what characteristics 
of a good teacher are, how can we select? I think if we want to 
shift that entire fallacy to the whole business of teacher prepa- 
ration we might say let's not do any teacher preparation either, 
because we don't know what a good teacher is. I think this is what 
we're going to develop in the next few days. 

Jay Greene: On Dr. Rivlin 's contribution, the plain fact 

is that selection is taking place - selection of teachers and selec- 
tion of civil service workers and selection of people for^ jobs in 
industry. It's being done because there are, in some situations, 
more people who want jobs than there are jobs. Now the question 
then arises if you are going to have to select - if you have 3 
people applying for one job, whether it's for teacher or principal 
or chairman, then what is the best way of selection? Dr. Rivlin' s 
assumption of throwing all the weight on the preparation and the 
post-appointment feature would be wonderful if you didn't have more 
people applying than you needed. Furthermore, it doesn't consider 
another practicality - that if you appoint somebody to a school in 
order to observe him in action, if you appoint the wrong person, the 
damage that may be done by this individual in a learning situation 
for a year or two is unfair to pupils. Looking back at Mr. Cogen's 
comment - he asks what studies have been made of our evaluation. 

One of the difficulties of such a study is that if you are going to 
compare what you believe to be a valid selection procedure with an 
invalid selection procedure, it means that you have to put the in- 
validly selected people into jobs so that you can then compare them 
to the people that you had validly selected . In basic research in 
publip administration this is morally wrong. How can you in con- 
science say, "These people we believe will make good principals. We 
will appoint them. These people we don't believe will make good 
principals but we'll appoint them for a study's sake in order to 
compare the good to the bad." This has been one of the great diffi- 
culties of this sort of research, although there are some others. 

I'm delighted that we're focusing on some basic questions that we 
seldom get to discuss. 

Gertrude Unser: I'd like to comment on one aspect of what 

Dr. Rivlin said. As I understand it, he was asking among other 
things, why there was not a greater emphasis upon an evaluation of 
past performance. The vast majority of the people who are selected 
for teaching positions have no past performance, unless we consider 
their college preparation as their past performance. Now I see two 
problems associated with selecting teachers on the basis of college 
records. First, we have some 1200 teacher preparation institutions. 
Those of us who have dealt with marks and recommendations have 
learned how unreliable these ratings and recommendations are, what 
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great variations there are. Standards differ widely. Second, al- 
though we confess that we have no proved means of validating selec- 
tion procedures against performance on the job, I think we have even 
less in the way of establishing the relationship between success in 
college preparation and success on the job. These two are quite 
different things. 

David Darland: I hope that we don’t lose track of what Dr. 

Rivlin raised, because I think he raises this as a process that be- 
gins in the teacher education program, if I heard it and reaches be- 
yond what we now consider to be the termination of the teacher edu- 
cation program. Is that what you’re saying? The selection process? 

Harry Rivlin? I’m sorry. I must not have said what I 
meant to say. I wasn’t thinking of the teacher preparation as the 
basis for selection, but I’m saying you have in a sense two dif- 
ferent groups. You have an experienced group and an inexperienced 
group. The experienced group consists of teachers who are coming up 
for administrative positions, or people who’ve been teachers or ad- 
ministrators in other cities. How can you evaluate past performance 
where there is no past performance? Where there is no past perform- 
ance, as in the case of a new teacher or a new principal, I wouldn’t 
rate him on the basis of his record in college. What I’m saying is 
that when you appoint him, remember that you’re appointing someone 
without a past record and you must evaluate the ” past” that he is 
creating in his classroom. I suggest that you select your new 
people as well as you can, but remember that you are selecting only 
risks, and that the important part of the evaluation comes at the 
end of the initial appointment. 

Jules Kolodny: I would direct my attention to several of 

the comments made here. First to Dr. Greene’s comment that society 
doesn’t want to set up a research technique by putting incompetent 
people into the field. We have school systems, (as we’ve been told), 
that have different kinds of selection procedures. It seems to me 
that we ought to determine whether or not the products (mainly the 
students who have been turned out by this institution called public 
education) are better, the same, or worse, depending upon the selec- 
tion procedures. To my knowledge, the colleges have made no such 
indication that those school systems that use a particular kind of 
selection procedure tend to give them better students than those 
that use a different kind of selection procedure. I haven’t heard 
this come from industry, either. We must remember that while we 
don’t have the intangibles and the emotional factors and possibly 
some of the citizenship factors involved in education, the fact is 
that in terms of the things that make youngsters employable, that 
make them eligible for academic training, I would asse* t in no wise 
depends on whether you lise one procedure or another. In other words, 
there are good schools that use procedure A and there may be good 
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school systems that use procedure B. It seems to me that we are try- 
ing to determine whether the procedures are sound in vacuo . If the 
procedures are sounds they *11 ‘turn out good products • If the proce- 
dures are unsound, they won't turn out good products. We have to 
follow the conveyor belt of what happens to the students who have been 
selected by procedure A, by procedure B, etc. I would assume that 
they get lost in the general review of our culture and society and 
the employment world, the industrial world, the commercial world and 
the collegiate world. Consequently, it may well.be (if I remember 
my undergraduate days and Mills Canons of Induction) that these par- 
ticular factors that we're focusing on are really irrelevant to what 
we're concerned about. 

What we are more concerned with is what goes on in the 
classroom. Therefore, this relates to what Dr. Rivlin has been talk- 
ing about. In other words, maybe we're setting up a straw man and 
we feel that we have to have certain kinds of procedures because 
traditionally we've used them. I do concede that the kind of selec- 
tion procedures that are not exclusively based upon subjective fac- 
tors will tend to eliminate discriminatory factors, in terms of race, 
religion, general attitudes and appearance, possibly. A system that 
relies exclusively on interviews may not have as many long-haired, 
teachers as the system that relies upon an effective written examina- 
tion$ but, I'm not so certain that this is the thing we ought to 
direct our attention to. Frankly, I'm not impressed by any of the 
research and I would not want to appropriate large funds for such re- 
search if I were a member of a Board of Education. I would look at 
one thing primarily as you follow the products of your school system 
which uses a particular kind of institution for selection: Are your 

products better, worse, or the same as compared to the products of 
other school systems that use different kinds of selection procedures 
for their staffs? 

Gertrude Unser: I'm addicted to responding to each speaker. 

But I think you say very truly that the ultimate, the end product of 
of all this, is the student and what happens to him. But certainly 
the quality of the teacher is only one of the many factors which 
forms the student. We are trying to find out what makes a good 
teacher and how a good teacher makes a successful student but to say 
that this is the only factor is oversimplifying it enormously. There 
are so many other factors which enter into a student's success. We 
have reached the end of the time originally allotted although I must 
say we did start rather late. Is there anyone else who has a question 
or comment? 

Bernard Berger: I wanted to avoid the questioning of the 

research done except that all I want to say is that I think what has 
been done is valuable and I think we should know preliminary to any 
further research the characteristics of school selection systems as 
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presented in the two studies 5 but I was a little unhappy, and 
relates to what Mr. Kolodny said, that there wasn't some attempt at 
the same time to find out something about community satisfaction 
with the selection systems used. For example, where you had a so- 
called primitive system did the results, the teachers obtained in 
this way, did they perform satisfactorily, was the community satis- 
fied, because up to this point of the research you know nothing but 
a series of facts, that there are various kinds of selection proce- 
dures used in various size systems. 

Harry Gilbert: All we did was to ask school systems what 

they say they do in terms of selection of teachers. If we would do 
it ovef again today we could do that little survey a lot better but 
ail we did was say“ "What do you folks do?" Now I'm sure we can get 
all kinds of leads for how that could be followed up ^ 

vital one, but at this very moment I have to remind all of us a 
all we want to do tonight is orient ourselves to the objec ^® s ° £ 
this conference and mainly recite to you, review for you, what we 
started to do. Let's save all this because I think every one 
these questions will come up in the course of the program laid out 

for the next day and a half. 



Paul Denn: 
is adjourned. 



Thank you, ladies and gentlemen. This meeting 



121 * 



ERIC 






Session No. 2s Personnel Selection Practices in Industry 

-procedures currently utilized 
-validity of selection procedures 
-relevancy cf industrial selection practices 
to selection of teachers 

Chairmans Jay E. Greene, Board of Examiners 

Paul Baker, Standard Oil Company of New Jersey 

Douglas W. Bray, American Telephone and Telegraph Company 

Paul Denns Good morning. Yesterday we did a little brain- 
storming and we did review some of the basic questions in connection 
with the selection of personnel; some of us asked, "Should we select?" 
Others, "Can we select?" Some others said, "Who should select?" And, 
finally, "How should we select?" We brought together representatives 
of industry, who select personnel for their purposes; civil service, 
and so on today. And this morning we'd like to hear about procedures 
that have been feasible or successful in both business and in civil 
service to see to what extent they can be helpful to us in the selec- 
tion of school personnel. To begin with, then, this morning, I'd 
like to introduce my colleague, Dr. Greene, who will, in turn, 
introduce his panel. 

Jay Greene: At one time, it was thought that the problems of 

selection in industry were remote from problems of selection of pro- 
fessional personnel in schools, and while this might have been true 
years ago, more and more the problems, as I reviewed them, seemed to 
be close together: close in their complexity. We often over- 

simplified their problems, I think, but their problems of staff se- 
lection are complex. I'd like to introduce our first speaker, Dr. 
Baker, who has a strange title for Standard Oil of New Jersey, Ad- 
visor on the Social Sciences for Standard Oil of New Jersey. I had 
to read that a few times because furthest from my mind was the in- 
volvement of Standard Oil of New Jersey in the social sciences. In 
this capacity, he does consult with managers of affiliated companies 
around the world on problems of selection, placement, evaluation, 
management development, sensitivity training and all the other good 
things that a good psychologist is interested in. Dr. Baker. 

Paul Baker: Thank you very much, Jay. I will try to follow 

the pattern set last evening in the beginning session by keeping my 
remarks ratHer brief, hopefully to the point and hopefully to open 
myself up for lots of pointed questions in an inquiry period to fol- 
low. As Jay indicated, I think that we will find in the course of 
these discussions that the problems of selection in industry are not 
materially different from the problems of selection any place else. 

I'm fairly confident that Standard Oil's selection procedures are 
not, in any real significant way, different from any other industry's 
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selection procedures. We all do pretty much the same thing. I 
choose to believe, however, that Standard Oil does a much, muc 
better job than any other company in its selection. 

The kinds of things we do are the kinds of things that you 
do in your teacher selection. We analyze our needs, try to antici- 
pate the numbers and kinds of people that we're going to need . We do 
it in terms of the immediate future and in terms of the long-range . 
future. We try to keep in mind at all times what our needs are going 
to be twenty years from now, as we consider our selection problems. 
Quite literally, this is true. We are thinking about the changing 
that's going on in our industry and the kinds of people a 
going to have to select today in order to have the right mix of 
talents and skills to man the industry, as it will be twenty years 
from now, jwhich is going to be considerably different in “any im- 
portant and fundamental ways. Having anticipated our needs, we g 
through all of the traditional steps of recruitment} we advertise ^n 
the journals, we scurry about the campuses, colleges and universities, 
trying to interest the young professional people in coming to 
for us. We even in some critical areas rather aggressively re 
in the high schools for secretarial help. This is particularly true 
in New York City, for example, where the clerical situation is tig 
at the present time. Having interested some young candidate m ap 
plving for work in our company, we in most all cases, an ere. 
exceptions to this, administer a selection battery, tests, quesion- 
naires, and again in most all cases these are fairly elaborate. test 
batteries. They are well -researched with proven validity and in 
testing programs both in the United States and overseas. 

I think this is the one place where we have concentrated 
and have been able to do real sound research, the kind of research ^ 
that would stand up as academic research, or thesis research, and in 
all cases, we find that we are able to develop selection tools or se- 
lection tests which do a very, very good job for us. And our test- 
ing programs, the tests that we have developed to the tests that we 
use, a catalogue of them is a fairly thick book. Admittedly, many 
of them, it * s the same test in all different languages of the world, 
but still they are different tests. 

We usually aim for a validity coefficient of at least .60 
and seldom fail; mostly our validities are around .70. I 
if we "were to distribute all of our validity coefficients, the. modal 
value would probably be around .6£, but we have many that run in o 
the high .70's which is good. 

We, of course, depend, as you do, rather extensively on the 
interview. We have done a bit of research on the interview process 
and we find that when we can get interview data cast ^to quantita- 
tive form that it too can contribute to our overall validity. But 
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this also depends upon the amount and quality of training that is 
given to the intervievrers and in many cases, in many of our instal- 
lations, we have trained essentially the whole of the management 
staff in selection interviewing techniques and in cases like this 
and in these cases where we can get good data from interviewers, we 
find that although the collective validity of the interview informa- 
tion by itself is rather modest, perhaps in the .30' s, this can add 
a significant amount to a statistical prediction, if you will. 

More importantly, and I think at a point there are many 
differences in the selection procedures we follow, deriving from the 
differences in the nature of the problems that we face, differences 
between industry and education, but one of the things that I think 
is a crucial difference is the fact that we consider the selection 
process not merely in the sense of selecting an individual once ; ad- 
mittedly, this is the crucial selection, that is the new employee 
who goes to work for our company, but there is a continuing process 
of selection from there on, and in many, many steps that one takes, 
one is faced with another selection procedure. That is, a craftsman, 
for example, is initially selected to enter training to become a 
craftsman in some future years and it may be a very brief period of 
time, four or five years, up to ten or twelve years, that this in- 
dividual will again be faced with the selection procedure for fore- 
manship training. And again, at this point in his career, he is 
submitted to exactly the same steps of being advised or recruited, 
if you will, of the opportunities, being interviewed, being tested, 
being examined rather thoroughly, and his work record then also be- 
comes part of the selection procedure, and this can continue on up 
into still higher ranks of management. 

The same thing is true to a lesser degree of clerical help; 
it is very true in terms of our selection of young professional 
people^ as they are being considered for management positions, they 
get probably the most exhaustive testing program of any in our com- 
pany in our management testing program. 

We are particularly concerned with the problem of select- 
ing managers and it's the one selection process upon which we h^ve 
concentrated in the last ten years, just now doing ten-year follow- 
up study on predictions that we made ten years ago about managers. 

We have done other follow-up studies, but not based upon this particu 
lar selection package and in our follow-up studies, we demonstrated 
that we do have long-range predictive validity; again, in the order 
of magnitude of something over .60, and we have extended this particu 
lar management selection program; we are in the process of extending 
it around the world, again in all languages. We've completed the 
study in Spanish in Latin America; we have in progress studies in Den 
mark, Finland, Norway, Sweden, Holland, and the Scandinavian area. 
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We have exhausted the study in Canada and in the United States, and 
have again found that as a selection procedure, this has a high de- 

gree of validity. 

I've mentioned the successive steps of continuing selec- 
tion as one of the major differences between business and industry, 
there are probably other differences between business and 
industry that would give rise to some differences in perhaps our em- 
phases -^that is, the broader opportunity that we have for placement, 
in otoer words, I think that we are able to achieve a greater degree 
of Dredictive validity in a statistical sense because we can accom- 
modate a much broader means of talent in our selection and when you 
are faced with the problem only of selecting a classroom teacher ^who 
is going to be a classroom teacher, as a career, rather than a. a 
person who is coming in with the idea that we are trying to select a 
good person for development and when we fail, in a sense, of finding 
fperson who is going to move up a particular line "e have lots of 
other lines in which we can put this individual. I don t chink y 
have quite that opportunity in educational institutions, so perhaps 
the glittering success that we had in our statistical studies of 
prediction are a function of the se differences. 

Well, with that kind of comment, perhaps I would like to 
start the question period by throwing questions out to you because 
I come here to learn as much as to try to inform, and direc ing 
questions to the people who are most concerned with teacher selec- 
tion- and these are the kinds of questions that we examine in our 
processes of consulting with managements. But there are the simple 
logical questions, like those of you who select teachers, do y 
ever make mistakes, do you ever hire people who turn out to be 
failures and, if so, how do you know? Another basic question, -he 
people who are responsible for hiring teachers, what difference does 
it°make if £u make a mistake? In other words, in business, when we 
pin down the accountability for mistakes and we can make i .stick, 
and we make it worth the individual's while to do a good job * e 
lecting people and he knows it, but does the same thing obtain in 
the education industry? But I would submit that if the^swerto 
the first question is yes, we know we make mistakes, and you can see 
these mistakes, then the opportunity most surely exists for you to 

do the^amTkiAds of research that we have done did ^e 

ing what are the characteristics of this good te^her.whydid we 
make this mistake and what we may do to avoid it in the future. 

Jay Greene: Thank you very much, Paul. We had agreed that 

both speakers would make their presentations before the 
oeriod I was interested, as I am sure you were in hearing Paul s 
comments on the difficulties of predi ct: validity and the prob lems_ 
concerning making mistakes, and that brought to mind this - . 
dote of a very young assistant professor who was teaching American 
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History in a university and he had in his class a wrinkled and 
gnarled old Chinese student who was there for a degree, w 0 ® 

philosophical bent of mind. This young history professor asked the 
class to name three good results of the American Revo _ut ion of 77 » 

^ he calS Sis elderly Chinese. The elderly Chinese got up and 

said, "It's too soon to tell." 

This is one of the problems of the predictive validity. 

T -] p „4. a teacher and it would seem, on the face of it, easy o 

say "He*s a good teacher, a bad teacher, depending upon - t ? e I ? r ° U ° 9 
i e measuring the students to see how well they do. If he 
eood ’ teacher ^ the students will do well." Then the anecdote comes 
to mind and you begin to ask yourself in what ways will you appraise 
Se^udSts 7 Their factual knowledge? Their changes of 'attitude? 

How do you measure this? Is it the immediate ef^ct °rwhat»ill 
happen to them in ten or fifteen years, or in a lifetime. All f 
this is complex, and troubles all of us who are here, including key 
people in selection in industry. That leads me to Dr . Bray, wo is 
theVrector of Personnel Research for American Telephone and Tele- 
eraoh Company, a grand national institution, ana whc prepared for 
S^posiUon 7 at quite a number of select universities. Doug Bray, 

you have the floor . 

Douglas Bray: When I prepared my talk for this Conference, 

I planned to present an overview of the various devices used to se- 
Lct“usiness managers, and their relative degrees of success, with 
+hP thought that you might see usefulness in some of them lor tne 
^obleiTof^teacher selection. As I listened to your discussion last 
evening, however, I was impressed that even in that short session 
w. 1 , had raised most of the crucial selection issues. It seemed 
2T therefore that I could make more of a contribution by addressing 
wseS to what I consider to be your central problem and focusing 
^ one management evaluation technique which may offer a solution t 

that problem. 

I recently had the pleasure of attending an excellent sym- 
posium organised byV Gilbert and Dr. Lang at the American Psycho- 
logical Association meetings, a symposium also on the^ subject o± 
teacher selection. One of the panelists. Dr. Elizabeth Hagen of 
Teachers College, read some quotations about the difficulties in the 
selectionsof teachers and the great need for -search Thesequoa- 
tions sounded very much like the remarks made in this ^ eV . 

ning and vet they were thirty to forty years old! My prediction is 
that unless vou face up to your central research problem, you can 
have meetings like this one for another twenty years without getting 

much further ahead . 

Tour central research problem is, of course, what psycholo- 
gists call the "criterion problem." Unless you can somehow agree on 
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methods for determining whether a person is a good teacher, an aver- 
age teacher, or a poor teacher, you might as well forego the rest of 
your discussions. In getting into this problem, I am probably going 
to impinge on the papers of some of the panelists scheduled further 
along in this Conference, such as those who are going to speak about 
problems of validity and the design of needed research. I hope you 
will forgive me if I go ahead anyhow. 

The management evaluation technique which I am going to de- 
scribe can serve as a selection method . This is in fact the way in 
which we use it. It seems to me, however, that there might be prac- 
tical difficulties in using the method in the selection of teachers 
which would be very hard to surmount. I'll touch on these later. 

More important is the probability that the method might yield a cri- 
terion. and if you could agree to accept this criterion the way 
would be open for research on the validity of the kinds of predictors 
used in the selection decision, such as interviews, biographical data, 
college grades, ratings in practice teaching, etc. 

We in business were faced with the same type of criterion 
problems as you are. Various appraisal plans have come and gone over 
the years, and yet few are ever satisfied that we can rate excellence 
as a manager with high .reliability. There is much evidence that or- 
dinary supervisory ratings contain substantial error. It is likely 
that many promising selection devices have been discarded because 
they were "validated" against poor criteria of success. 

We believe that we have found at least a partial way out of 
this dilemma in the technique we call the "assessment center." We 
developed this method by first asking higher management to agree on a 
number of characteristics which would usually distinguish between 
good and poor managers. After much discussion and definition and re- 
definition we wound up with a list of 2* qualities. Among them were 
such things as oral communication skill, decisiveness, leadership 
ability, independence, etc. You could probably continue the list 
yourselves and come pretty close to what we have. 

After these qualities were agreed upon, we devised tech- 
niques which would elicit behavior relevant to these characteristics. 
These techniques included interviews and paper and pencil tests, but 
more important are the "simulations." Usually there are three of 
these. One is an individual administrative task, known as the In- 
Basket, a technique, incidentally, which has also been used in. the 
study of school administration. The other two are. group exercises— 
a miniature business game and a leaderless group discussion. The 
total assessment runs for three days, and twelve candidates at a 
time are assessed. 

Assessment staffs are carefully trained to administer the 
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techniques, observe behavior, and rate the 25 management character- 
istics^ After a candidate is rated and thoroughly discussed at a 
staM meeting, the staff places the candidate in one of four cate- 
gories • "more than acceptable," "acceptable," "less than accept- 
able," and "unacceptable." Staffs show a high degree of agreement 

in this final judgment. 

Our first assessment center which actually operated in the 
selection of management opened in 1958. We nowhaveoverg centers 
and about 25,000 men and women have been through them. This growth 
has taken place not because of any high level decree but because 
local^ine management observed the process and judged the method to 
be^even though comparatively expensive, a highly valuable tool in 
the selection of better management. I might note that the assess 
ment staffs are not professional psychologists. They are, by and 
iSe! seconSlevel Lnagement people, most of whom have not attended 

college . 

I would like to propose that you develop an assessment cen- 
ter for the evaluation of teaching ability. The qualities looked for 
would, of course, not be the same as ours and the a3 3 e3 3 ">ent exer- 
cises would most likely be different, but I have no doubt that it 
would be quite easy to arrive at highly satisfactory me os. 

Such an assessment center could, in theory, be used in the 
actual selection of new teachers, but there might be insurmountable 
Dractical difficulties. The great number of candidates, most of 
whom have to be evaluated at the same time of the year, 
staffing imDossible. Assessment might, however, be done during the 
finaf vLr ta schooi or during the new teacher's probationary period. 
I don't Snt to close the door completely on the use of assessment as 

a selection method. 

My definite recommendation is, however, that the assessment 
center be used as a research criterion againstwhichto ™lidatethe 
techniques and indices which are used, or might be used, m 
tual employment situation. The profession would agree, in effect, 
that nerformance at the center would be taken as evidence of g°°d, 
medio^e, on°or teaching ability, and selection research could then 
get going in earnest and with the hope of early an grea 

I know that many in this room are already inwardly protest- 
ing You 'relaying that the criterion of good teaching is whether 
the 'children taught are effective and happy citizens years after 
toey-reoutof school, and you think, I am sure, of any number of 
reasons why you wouldn't buy anything less than some ultimate cri- 
ter ion 3 like that. Well, if you won't, you might as well stop having 
conferences on teacher selection research because you aren t 
to get any productive research done You must agree on ^me J 
and convincing criterion which it is practical w gcu 
center can provide such a criterion. 
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Discussion 



Jay Greene: Thank you very much Douglas. I have always 

been an admirer of the ingenuity and the originality of people in 
business, especially since an experience I had about a year or two 
ago in connection with some editorial work. They were talking about 
the book used in elementary school. Look, Dick, Look . All of you 
are familiar with that, and they wanted to modernize it. There was 
a meeting of some professional advisors, and they decided to take 
the book. Look, Dick, Look, and modernize it by spelling it back- 
wards. And if you spell Look, Dick, Look backwards, you get an in- 
teresting title, Kool, Kid, Kool . Since that time, I’ve been an 
admirer of the ingenuity of business people. We have some time, 
now, for questions, and we would be delighted to refer them to our 
two speakers. Yes, Al. 

Albert Schiff: The question I would like to ask Drs. Baker 

and Bray is a question relating to the selection of people at the 
time of initial appointment. At the present time, you were talking 
primarily of management people; these are people already in positions 
with your Company, who are working up to management positions. For 
example, teacher candidates are graduates of colleges and univer- 
sities. When you go recruiting in colleges and universities, look- 
ing for engineers or chemists or business-ad majors, do you put them 
through a testing procedure? Or do you say, he’s a graduate of MIT 
or a graduate of so and so; in other words, what do you do for 
people at the professional level who are not in your employ? 

Douglas Bray: Yes, we do test college graduates from even 

the best colleges before employment although we don’t attempt to put 
the potential recruit through an assessment center. We could not 
possibly staff up to assess all of our college recruits on a pre- 
employment basis. We are now, however, putting many candidates for 
sales employment through a pre-employment assessment center. 

Paul Baker: I would answer the same question in terms of 

Standard Oil. We do not universally test young professional people. 
In certain functional categories they are tested prior to employ- 
ment. In the total sense of all of the people hired during the year 
in the United States, we would be testing about half, prior to em- 
ployment, with specialized testing instruments, not a massive test- 
ing battery, but more or less a screening test. We do, however, 
anticipate, in the not too distant future, almost complete testing 
of young professional people. 

i 

Milton Gold: I was particularly interested in Dr. Bray’s 

suggestion there that use might be made of some kind of assessment 
in the student’s undergraduate work and in the first year of teach- 
ing and I recognize some kind of relationship here to Dr. Rivlin’s 
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remark last night that the rating at the time of initial employment 
mieht rather be considered as a screening device which is followed 
up in the tenure period. I think that teaching has certain advan- 
tages over industry in terms of a closer relationship with the 
undergraduate preparation than is true in industry, and that it s 
possible for the New York City schools, for example, to have a very 
close relationship with colleges in the Metropolitan area, in terms 
of efforts to do some evaluation prior to graduation. And that it 
might be possible, then, to spread out this testing of the student 
and to get a broader sample than is possible than on a one-day 
written test plus an interview and any other kinds of performance 
that are required, so that maybe Dr. Bray's suggestion here of ob- 
servation in the teacher education days would give a far broader 
sample once the criteria have been established. 

Fred Williams: I wanted to ask Dr. Baker, first, the 

nature of the training that is given to the interviewers whom you 
spoke about and then ask Dr. Bray if he would also comment on the 
nature of the training given to the staff of the assessment centers. 

Paul Baker: The nature of the training of, well, let me 

just give you as a case in point, I'm sure, and I do this because 
I'm sure you'll recognize the name Anne Roe, for example, as our 
principal consultant in interviewer training. She's had a lot of 
experience in interviewing and a lot of experience in teaching and 
in the field of psychology. And her ^veiopment program is struc- 
tured around first a study of all of the other kinds of information 
that one has available at the tdine of interview or prior to an in- 
terview, such, so that primarily you teach an interviewer not to 
attempt to make judgments about facts about an individual tha ar 
more readily obtained some place else, for example, intelligence. 

Why should an interviewer waste a lot of time trying to assess the 
intelligence of an individual when a twenty-minute test will do a 
much better, more accurate job. 

But related to intelligence, a thing which one cannot so 

readily assess with a paper and pencil device is I* 1 ®' ^ence 
one is going to use that intelligence, or how well that intelligence 
is applied in a practical sense. So the interviewing is, the train- 
ing is one of an intellectual sort that is a study of what is appro- 
priate in an interview, followed up by training, actual interviewing, 
tape recorded and fed back and an examination of the mistakes that 
one makes and an attempt to correct these mistakes; and it s an in- 
volved process and a fairly involving process. 

At the present time, we're doing some training, using the 
little video recorder, which is just extending the same technique m 
terms of the technology available, and I think that you 11 
the future rather widespread use of the video recorder in trai ng 
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exercises, and particularly of this type, where you can get an imme- 
diate playback and see yourself, you know, and as well as. hearing, 
you can see and hear the kinds of mistakes that you make in the train- 
ing. Does that answer your question sufficiently or would you like 
to get into some of the details? 

Douglas Bray: The training of the assessment staff occupies 

as much as three weeks; generally, we recommend three weeks, and the 
training amounts to, well, learning how to run the techniques, of 
course, but that's minimal. The important thing is to develop a com- 
mon understanding of the definition, the meaning of these 25 charac- 
teristics, writing reports, so that others can see what you saw. 
through the report, because not every observer sees every exercise, 
obviously, writing reports to convey to the other members m ^the 
group what happened, so that they can also make inferences about the 
behavior and “arrive at the ratings. The last part of this three- 
week training is a dry-run group, a practice group. for the staff to 
try out their wings before they meet the real candidates. 

Thomas Bransford: I'm prepared to believe that Standard 

Oil has a very fine selection program, maybe the best in the world, 
but I am suspicious, or let's say curious,. as to the operations that 
will lead to a predictive validity coefficient of .75>* However, I 
can imagine some. I can suggest, for instance, that if I relate the 
proportion of admitted mistakes in hiring to the total number hired, 
and that I detect and admit one mistake in ten, "Yes, boys, we 
shouldn't have hired him." Then I compute a shortcut phi coe 
ficient by subtracting the percentage of mistakes from the per- 
centage of satisfactory new-hires, then I get .90 minus 10 and I come 
up with a synthetic coefficient of .80. However, if I follow up re- 
jects or if I correlate some scores on selection tests with any kinds 
of ratings or any kind of objective criteria afterwards, and I con- 
sider that the practical ceiling on any one correlation coefficient 
is going to be the product of the reliabilities of the test. and of 
the criteria, I just can't expect predictive validity coefficients 
much above .60, even under very favorable conditions. 

Paul Baker: That's a complex question, the kind of ques- 

tion I used to get from my graduate students and I would send them 
back and say, "You reduce that to one I can answer yes or no and then 
I?ll talk to you about it." Most of the time they'd never come back 

with the same question. 

You're asking, essentially, how do I dare stand up in front 
of an audience and claim a validity coefficient of .7£ *nd how do we 
do it? And I don't know how deeply to go into this.. Obviously, we re 
concerned with the criterion problem, and Doug has discussed it in 
some of his earlier comments, and we have devoted probably more effort 
to the sSdy of the criterion problem than.we have to the selection 
problem or the prediction problem, prediction instruments. As you 
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yourself point out, you must have a reliable criterion measure before 
you can get anything to correlate with it, but having found a reli- 
able criterion measure in a statistical sense, say, an effectiveness 
in some kind of performance on a job, then it’s relatively simple to 
pick up a handful of instruments that will scan the mental abilities 
and some temperament factors and get a very good coefficient. 



What we do in general and in our concurrent validity studies 
is to approach the problem of the criterion, not directly, and this is 
an argument that we have, it's almost a continuing dialogue with our 
superiors in the organization. How do we know that you are predicting 
success, and what is that, really, because everybody recognizes that 
success in business is a complex concept; success is composed of lots 
of different things, and we say, well we don't really, but what we 
look at and what we're trying to predict and for purposes of research, 
let's accept the fact that we look to the intervening variables of 
present and past rewards for successful behavior in a rather gross 
organizational sense. 

In a typical study, and these have been done many, many 
times, and for many, many years we've been doing this, we pick up 
variables relating to height, relative position in the organization, 
salary history, evaluation histories, functional histories, and so 
forth. For example, in one of our major studies, we had 72 bits of 
criterion information, things about people that reflected, in some 
sense, successful behavior. This matrix was factored and the first 
components had to be adjusted just a little bit to throw it through 
the age factor, so that it would be correlated zero with age and, 
using this as a first principal component, we look at the structure, 
does it make sense? If it makes a lot of sense, this then becomes a 
criterion number, such that you can get a score for each individual 
on this criterion variable and then you look at people who get big 
scores and everybody will agree, yes, if we could get more people, 
like that, we'd be happy. And this becomes a criterion number which 
then becomes, with a high degree of a kind of statistical reliabil- 
ity if you will buy the concept of the length of the vector as in. 
some way reflecting the reliability, becomes a vector variable which 

we can predict. 

Other procedures that we've used have been the canonical, 
variate. It's pretty much the same sense, that is you have a matrix 
of criterion information, a matrix of predictive information, and 
you, those of you not familiar with the canonical model, you essen- 
tially factor both of these matrices and then fit the two spaces 
together to maximize the correlation between them. 

I've done a couple of studies where I've factored the cri- 
terion matrix. I've calculated the canonical variates and I've done 
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a thing which I call iterative multiple regression, , and this is 
striking a trial vector on one side and regressing one set of vari- 
ables against that and then taking the regressed variable and re- 
gressing the other against that until the thing stabilizes. I 
haven’t worked out all the analytics of this yet, but I know it con- 
verges not to a first component and ndt to a canonical variate, but 
to, in a sense, a more practical variate. And again, we come up 
with a criterion number which, people will agree, yes, that, the 
people in this bunch here with high scores are the kind of people 
we like, and they’re successful people, and these others are not so 
successful people. This becomes the thing that we can then compare 
with all of our predictive variables. 

Although I claim only validities in the .70 's, in many 
cases, and these are cross- validated in all cases, in some cases 
right now we’re doing a study where we have s5.x basic groups which, 
and that’s cross-validation with a vengeance, if you will, and not 
concurrent validity but long range, well five-year validities. I 
expect to get, again, validities, in the high .70's here. In some 
studies, particularly when we have a well-researched battery of in- 
struments and we take it into a foreign language in a foreign coun- 
try, we find that almost invariably we do a better job there, the 
validity is higher than it is in the United States, for a variety of 
reasons; you’ve got a much wider range of talent, to start with, and 
our foreign friends, being less sophisticated in these areas, do ex- 
actly what we tell them to do, so we get very, very good clean data, 
and they work very hard at producing good data for us. Does that 
tend to answer the question? 

Thomas Bransford: In a way, as you did in your paper and 

you alluded to in your speech, the idea that you don’t regard the 
selection as a one-time, ten-minute process... 

Paul Baker: No. 

Thomas Bransford : ... but as an accumulation of data over 
a long period of time. You also said that you have the advantage of 
multiple placement . . . 

Paul Baker : Yes . 

Thomas Bransford : ... so I assume that part of your success 

is that you use a lot of information that we’d ordinarily call cri- 
terion information, and say, well, yes, maybe we made a mistake in 
his initial placement, but, gee, now, now we know what he can do, 
what he’s good for, and before... 

Paul Baker: No. 
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Thomas Bransford: ... you get through, you don't have too 

many people you can't use somewhere, somehow. 

Paul Baker: We attempt very strenuously to avoid the con- 

tamination of predictors. We are looking at, always the basic ques- 
tion, what is it that we can determine ahead of time, without any 
contamination of the criterion variables with the predictor vari- 
ables? If you would examine our research in detail, I think you 
would appreciate the fact that we do avoid this contamination. 

One of the reasons we're doing this five-year follow-up 
study now on this rather massive bunch of data involving some 3 ,000 
cases is that we've been using these data in the developmental proc- 
ess of a number of these people and we're just at the point now 
where the data that we have are uncontaminated, but they won't be 
very much longer. It's going to be getting at - in other words - 
things have happened to people as a result of the predictions that 
we have made. It's happened at a very high level in the organiza- 
tion, so that the contamination has been kind of indirect. We will 
never be able to use these data in the future for any kind of 
statistical studies; in other words, our validity is going to go 
way up, because of direct contamination. 

William Brown: I'd like to go back to Dr. Schiff's ques- 

tion and ask a little more from Dr. Bray on initial screening and 
testing, after campus recruitment. The situation. I 'd like to stress 
here is, you're a recruiter from a large city school system, you're 
critically short on teachers, you're at a university placement of- 
fice and you're interviewing candidates and most likely in the next 
three or four rooms are interviewers from surrounding attractive 
suburban communities. These interviewers offer contracts, sometimes 
based on a fifteen-minute interview. As a representative of a large 
city, and remember the teachers are selecting the city, and it's 
very difficult to recruit them, what testing would you do in that 
connection? 

Douglas Bray: If you are really in a situation in which 

you have very little opportunity to select at employment, which is 
the one you have outlined, then I guess I would not try to select at 
employment. I would try to select by attrition during the first 
year on the job. We have found, however, that interposing a short 
adjunct to selection, such as a mental ability test, does not inter- 
fere with recruitment. It may be, in fact, that making a job a 
little harder to get may cause more people to be interested in get- 
ting it. Even if some school systems cannot afford the luxury of 
selection at the moment, I still believe that research into how to 
select better teachers is still of great importance. 

Rufus Browning: We recently had a change in our rules, 
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which recognizes this factor, just brought up. We now permit a 
teacher to become a probationary teacher without a Passingscorexn 
the National Teacher Examination, with the proviso that the teache 
co min g aboard sign a contract to take and pass the National Teacher 
Examination during the probationary period. Failing to pass 
test during the probationary period is automatic termination at 
end of the period. But this is in recognition of the point just 

brought up. 

Perry Kalick: I'd like to point out that the staffing of 

the A.T. and T. assessment centers constitutes time, and that time 

is money. . . 

, Douglas Bray: We're wasting your money, as a stockholder? 

Perry Kalick: No. There's no question of the worth of 

sDendine $300 to $500 per candidate, if you^ave potentially a 
$100,000 to $200,000, or more, investment irts^jcandidate. ^arge 
city school systems spend only a small fraction of the, perhaps, 

$300 to $^00 that business and industry might spend . I would like 
to follow up on Dr. Gold's comment on assessing the undergraduate 
training of a teacher prior to entrance. I've ^en supervising 
student teachers for two years in East Harlem, in the types °* 
schools to which eight out of ten of beginning teachers are appointed . 

The basic requirement is that the college supervisor ob- 
serve the student teacher four times a semester. I have functioned 
more as a "resident" in this situation in that I have devoted a con- 
siderable amount of time to the training of thesestudentteachers; 
i.e., I have observed each student teacher about twenty times in 
varying situations, including two full morning observations with the 
cooperating teacher out of the room. 

I have found that student teachers who have undergone this 
extensive experience in reality are mostly rated considerably above 
average in their ability to cope with the problems that Peseta 
finin g teacher in a difficult school, by the principals of the 
£ES to which they are permanently appointed. I go along with 
AT and T.'s assessment center simulation approach. However, in 
the* college situation there should be an opportunity to assess a 
student's teaching performance prior to the upper or even lower 
senior year. We need an early indication of a student s_ teaching pe ■ 
formance so that we can tailor subsequent teaching experiences in 
accordance with a student's strengths and weaknesses, or perhaps 
have a basis for not admitting a student to student teaching. Fo: 
some students one semester of student teaching may be adequa e, or 
others, an entire year is necessary; and some students should never 
have gotten into the formal student teaching course. IJm suggesting 
that the college has a prime role in the initial screening phase of 
the teacher selection process. 
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Jay Greene: Thank you. I'm .afraid that the time allotted 

to our panel has been used up and I do want to thank Douglas and 
Paul for their presentation from their background in industry. We 
are fortunate in having around the table people with varied 1 experi- 
ence and .background, and we hope that this panel represents the be- 
ginning of a sharing of ideas, a give-and-take in discussion, in 
which all of us will be prepared to, perhaps, make changes in some 
of qut basic 'thinking, and perhaps, chart new channels or new ef- 
forts. This is one^Sf the most difficult things to do, to change 
basic thinking because everybody believes that- what he's doing and 
the way he is doing is the best w^y. We hope that the 1 two speakers 
here from industry will continue to share their thinking with us in 
all the problems that the panels will bring to us, and we hope that 
you'll participate as freely and as honestly and as delightfully as 

you have this morning. Thank you. 

* 
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Session No. 3: Personnel Selection Practices in Civil Service 

-procedures currently utilized 
-validity of selection procedures 
-relevancy of Civil Service selection procedures 
to selection of teachers 

Chairman: Samuel Streicher, Board of Examiners 

Bernard Berger, Department of Personnel, The City of New York 
Thomas L. Bransford, New York State Civil Service Department 
Raymond Jacobson, U. S. Civil Service Commission 



Paul Denn: Could we ask you now to sit down, please, to re 

commence. Someone in the lobby reminded me of an old story about our 
mistakes. He pointed out that doctors bury theirs, lawyers bill 
theirs, and we promote ours. At this point, let's turn now to the 
practices that obtain in the various branches of Civil Service. ^ I d 
like to introduce a colleague. Dr. Samuel Streicher, who 11 chair 
this panel. Dr. Streicher. 

Samuel Streicher: I think that one reason for meeting far 

away from home is that we have an opportunity to relax and listen to 
what the other fellow suffers, find that his problems are very much 
like ours, and from it derive a partial consolation. We also get a 
chance to relax in attitude to the point where we're willing to en- 
tertain another fellow's point of view, and even to listen to the^ 
way he goes about doing things. And in this process there may arise 
a still small voice that questions one's own practices, the assump- 
tions on which the practices are built, and, later on, urges us to 
entertain some modification of them. This self-reappraisal is a 1 
the more important in connection with the selection of personnel in 
Civil Service jobs, whether they are at the Federal, State or local 
level, whether they are in teaching or in non- teaching areas. 



To make the transition from industry to teaching under 
Civil Service is quite a leap. You first must move from industry to 
teaching in areas in which in the selection of personnel there is no 
control other than a very vague one. Thus, the Superintendent of 
Schools of a given community will determine for himself how he 11 
select teachers, principals, directors of supportive services, etc., 
and if he asserts that he has no examination but merely interviews 
somebody, this person, as Harry Gilbert points out, has received his 
examination, though he doesn't know it. The Superintendent is not 
accountable to anybody but himself and his Board of Education ; and, 
fortunately for him, the Board of Education doesn't hold him ac- 
countable, in the usual sense ; if he made mistakes, well, that s too 
bad . The mistakes will eventually wash out after 35 or more years . 
of service, unless the person is accommodating and quits or otherwise 
separates himself from service earlier. 
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When you come to Civil Service, with Federal or State and 
City laws setting boundaries, and when it's the policy of the people 
to make your decisions reviewable by others (usually by courts or 
commissioners of education), the temptation arises to create elabo- 
rate machinery, to become efficient in its operation, and then to 
find it downright uncomfortable to make changes in procedure and 
agonizing to change underlying assumptions and general design. 

There's the temptation to continue what we've been doing, and to be- 
come victims of our own procedures. That's another reason why get- 
ting together here, not only with representatives of industry but 
also with representatives of Civil Service groups at different lev- 
els will help each of us. 

I assume that when we talk about the selection of teachers, 
we are talking in generic terms; thus, teachers would include super- 
visors, administrators, directors of special service branches. Some 
reference was made. earlier today and late last night to the selec- 
tion of people at the upper reaches. Second, we include within our 
consideration of the selection process the initial screening as to 
competence and the appointment policies that may or may not facili- 
tate employment of the better people and may keep many a fine person 
from even making application. I believe that we also imply that re- 
tention policies determining the conditions under which people work, 
particularly with regard to morale (so badly abused a word), will 
come within the purview of our discussions. It will be easy for any 
one of us to defend himself and say, "That'S. the law." Well, first 
of all, we're not so ‘sure that we know what is the law; even those 
of us who are in the know, are sure of one thing: we don't know the 

law fully; and second, laws can be changed, for they merely represent 
the policy of the people at a given stage and the policy may change. 

Thus, as we proceed to examine ourselves we may think of 
the Biblical dictum, "Let us come and reason together," or more 
nearly literally translated, "Come let us become wise together." 

The latter connotes a receptivity, a mutual impact, and a hope that 
the spirit of exchanging, of willingness to listen, will drive away 
any temptation at defensiveness. 

The three speakers for today have consented, under the dic- 
tation of the Chairman, to limit themselves to about ten minutes of 
presentation. They will not read their papers, though they prepared 
papers, then we'll open the meeting to discussion, not merely to 
questions addressed to the speakers but to a free-for-all, a tearing 
apart of the ideas presented. The first speaker will be Mr. Bernard 
Berger, who is Director of Training in the Career Development Divi- 
sion of the New York City Department of Personnel. Formerly, he was 
Principal Personnel Examiner of the Bureau of Examinations of the New 
York City Department of Personnel. Mr. Berger. 

Bernard Berger: The selection of teachers and the other 



Dositions in the echool system generally involve approximately,] per- 
hat)s a maximum of about 100 different kinds of positions; in the 

Civil Service generally, City, State and ^ tl0 “ e ™ 

the different kinds of positions - number in the thousands. ¥e run 
the gamut of positions from those of cleaner, porter, laborer - 
whose starting salary is about UO per cent higher than that of a be- 
Snf™rtaNew York City - to positions which require, as a 

minimum, a Fh.D. and a certain number of years of experii » . 

_ rrtn sjfifi tjo have a very wide variety of responsibilities in sei.ec 
tion. The selection Sthods used in the New York City Department of 
Personnel don't vary very much from those that are used in school 
teacher 8 selection^in administrative selection, or in «»e selection 
methods used in private industry that were described eariier today. 

We have attempted and experimented with all of the kinds °*£®sts 
that have been devised and developed by ourselvesa^byothersin 
trying to select those who will be able to serve the C^ycompetn 
lyTperhaps creatively. We find that we have the 
teacher selection organizations haves they are not very much dif 
ferent. We've come to the realization that we have to experiment 
aS we are experimenting and we have changed many of our methods oi 
Gaining people quickly in shortage areas, of taking less than we 
normally^expect, and of consulting with every possible resource to 
"etter selection method - a more appropriate selection method 
that will fill the needs, the staffing needs of the City. I m n 
ffoine to talk in detail about the kinds of tests we do employ. I 
lust^want to speak about the validity of our selection procedures, 
which was one of the areas in the assignment given to us in prepar- 
ing for this conference. 

(Mr. Berger now read his position paper which is repro- 
duced on page 26 .) 

Samuel Streichen Our next speaker is Dr. Thomas L.Brans- 
ford Director of Examinations, New York State Department of Civil 
Service Dr. Bransford formerly was Chief of the Test Developmen 
Section of the United States Civil Service Commission and has been 
resident of the Personnel Division of the New York State Psycho- 
logical Association and currently is President of the Psychologies 
Association of Northeastern New York. Dr. Bransford. 

Thomas Bransford: We can tell how we do things and how 
aood the way we do things is for our purposes, and still miss tne 
work- comDletely with respect to improvement of teacher selection. I 
had^exoected that my prepared paper would have been distributed to 
efer^y Nowl Snd it has been distributed only to the other 
panel members and that we do not have time to read ^ 

Daner tells the framework in which we do our work, the variou g 

we try to do, our chagrin at how little we know about how well we are 
doing^it, and questions its relevance to teacher selection. First.I 
w?ll skto the paper and then I will go beyond it to speculate on the 
things I think are wrong with teacher selection. 
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If we were talking only about the comparison of various 
kinds of Civil Service approaches to Civil Service problems, I would 
have a much simpler topic. I know and could explain some differ- 
ences tetween the State system and the Federal system, and the City 
system and I could explain some differences in the American Civil 
Service system and various foreign Civil Service systems, many of 
which are based on accidental differences in law, differences in cul- 
ture, differences in kinds of problems. 

Basically, it’s almost impossible to summarize what we do. 
We're very much like the octopus, who was asked how in the world he 
managed eight legs, and, of course, he was unable to move, pondering 
that problem. We are responsible for holding U^OO separate competi- 
tions, separate examinations in the course of a year. These are for 
some 3000 classes of positions in the State service, and 10,000 dif- 
ferently named positions in municipal services, which probably repre- 
sent only several hundred really different classes. Our main con- 
cern has been how not only to meet the objectives of a public system, 
which is open to review and challenge by many individuals and inter- 
ested groups, but how simply to keep ahead of such a tremendous work 
program. 



Historically, the American Civil Service was based on a 
so-called competitive system, in which it was assumed that public 
jobs were in great demand and that many people were competing for 
the choice plums. Under such a set of conditions, very elaborate 
evaluative techniques might be appropriate, but as Superintendent 
Brown from Los Angeles pointed out in his comment in the previous 
session, one of the facts of life is that we are in an entirely dif- 
ferent kind of labor market. We have great scarcity, and our prob- 
lem really isn't the refinement of our evaluation techniques, which 
could be much more refined if we had more resources and more time. 
Under present conditions the competition is not among candidates for 
jobs but among employers for quality personnel. 

There is an element in American Civil Service which differs 
a little irom foreign Civil Service systems and from industrial sys- 
tems in that tne central evaluation agency does not complete the 
selection process. It nao the task of coming up with a panel or a 
list of qualified candidates in a one-dimensional sequence, supposed- 
ly binding and relevant to filling any individual position in that 
general position class. One safeguard we have, and it is really not 
so much a matter of any recognized inadequacy of our testing program, 
but a matter of constitutional law. In most Civil Service systems in 
the United States, under the American theory of checks and balances, 
the Civil Service system is regarded as the creature of the legisla- 
ture whereas the appointment process is regarded as the province of 
the executive branch. Attempts, in Civil Service practices and 
rules, to limit selection to the single highest ranking eligible 



have run against this Constitutional doctrine, the separation of 
powers, so that most Civil Service Agencies now allow an appointing 
officer, a representative of the executive branch, a choice from a 
limited number of the eligibles ranked highest for a particular 
class of jobs. Many jurisdictions have a rule of three. This rule 
of three is under attack, on the one side, by one set of pressure 
groups, and on the other by other pressure groups. I’m reminded of 
Luther Gulick’s aphorism "Opposing pressures keep public officials 
upright." Our employee groups and our minority groups, generally, 
would prefer a much more rigid system, whereby there was no choice 
by appointing officers. Appointing officers often feel that they 
should be allowed even more latitude in selection from an eligible 
list according to their appraisal of the candidates in relation to 
their needs. 

Mr. Berger has opened up this subject, so I guess it’s 
fair to pursue it, that the promotion racket in the New York City 
Sanitation Department’s appointments did not reflect at all on the 
evaluating agency, but on the appointing officer’s choice among 
groups of three. Minority groups and reform groups often have taken 
the position that this choice of three, or proposed extension to 
five, does give an appointing officer a chance to play various kinds 
of favoritism with the class tie, or with the political affiliation, 
or in this case, something much more crass. 

However, we can discuss the climate, and the conditions 
and the problems of evaluating people for the public service gen- 
erally and still miss the mark completely with respect to the teach- 
ing profession. I’m reminded of "10,000 Careers," a study made by 
Dr. Flanagan’s colleague. Dr. Robert Thorndike, in which 10,000 
people who’d been tested in the Air Force Selection Program were 
followed up in civilian life. Surprisingly, those who ended up as 
teachers were characterized by low verbal and quantitative scores 
on aptitude tests. If we follow the reasoning we use in validating 
interest inventories to find more people like those who are already 
in a profession, we might select for teacher training, candidates 
with low scores on verbal and quantitative aptitude tests . But 
should we be bound by the kinds of people we now have in the teach- 
ing professions? Perhaps we have a real shortage of people who are 
interested in the conditions of employment that apply to teachers. 

I would surmise that New York City has more problems with the black- 
board jungle image, whether it’s a fact or not, as a deterrent to 
teacher selection than with possible inadequacies in the selection 
instruments. Until New York City teaching jobs can be as secure as 
the suburban schools are, it’s a losing race as Superintendent Brown 
has pointed out to have more time-consuming and more elaborate selec 
tion procedures. What we must do is a bit of social planning. We 
must assess teaching potential, make the teaching profession more at 
tractive to young people, and guide people with the potential for 



Ikh 



teaching success into this profession. When we have done these 
th“ there will be more point to further refinements in the evalu- 

ation process. 

To avoid misunderstanding, however, I want to make it 
clear that I don't mean to say that careful selection methods ha. 
place oSy In a surplus labor market; I think the reverse is just as 
true. When we have a shortage of fully qualified people, fully 
qualified in the sense of meeting all our desired requiremen so 
education and experience, we still have a need to determine ap titude 
for on-the-job training and to direct likely prospects into training 
programs, possibly the type that Mr. Berger talks i about, sidestep- 
ping or postponing certification requirements. When we in thepub- 
lic g service got away from the idea that the only Person worth hiring 
was somebody who's been let go by another employer in the same type 
of work, we not only greatly increased the numbers of candidates but 

materially improved the quality. 

Mv personal experience with required education courses is 
limited and unfavorable. Of 2 C's I received in college one was in 
aneducational psychology course taught in theEducationOepartment. 
The psychological content of this course was obsolete and conflicted 
with more modern information from the psycholop- courses I was ta - 
in? in the Psychology Department. What were the right answers in 
the Psychology Department were not recognized in the Education De- 
partment. I used the Psychology Department answers a^ did poor y 
on the final examination in the Education Course. The other C in my 
college work was based on two tests I can neverforgetThecourse 
was in the psychology of philosophy and was taught by Edwin Ray 
Guthrie, a leader in psychological theory and later P^sidentofth® 
American Psychological Association. The course covered the psycho- 
logical content of the classical philosophers from Plato and Aris- 
totleup 0 through Descartes and Leibnitz. The first part of the test 
required identification by author of 17 passages and since we ha 
studied the works of 17 philosophers, the students tended to a - 
tribute each passage to a different author. The correct answer was 
Plato 17 times. The other part of the final course examination was 
a true/false test in which we were to leave atone the statements g 
that were true and to cross out the item numbers of those J 
false. Most of us marked about half the statements false but the 

key was all statements true. 

Samuel Streichers Our third speaker is Mr . Raymond Jacob- 
son, Deputy Director, Bureau of Recruiting and Examining, United 
States Civil Service Commission, in Washington. Formerly, he was a 
member of the Division of Standards of the Civil Service Commission, 

in Washington. Mr. Jacobson. 

Raymond Jacobson: I think I would like to use the limited 
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time available in order to permit the discussion to get under way, 
lust to touch on three or four areas, not to really summarize my 
whole paper again, which you will get. One thing I would like to 
point to, in connection with the Federal Civil Service, in attempt- 
ing to distinguish it from other Civil Service jurisdictions (be- 
cause I think there are some differences, although there are a great 
many similarities) is that the Federal Civil Service Law as ° ne 
tremendous advantage in that it is a very broad and general piece of 
legislation. In fact, we're convinced that the authors of this leg- 
islation, back in 1883, were extremely wise men, even though I ye 
never heard of Mr. Pendleton's doing anything else in his stay in 
Congress. In this area, they wrote a law which was really a very 
broad statement of general public policy and basically this is the 
legislation we still are able to live with. It has made things 
easier in many ways, I think, than in some state and local systems, 
where they have been harnessed with much more rigid legislation. 

We do have a broad statement of public policy, which is basic ally 
that the purpose of the Civil Service System is to see to it that 
jobs are filled on the basis of merit, that the details and the pro- 
cedures by which this will be accomplished are determined by rules 
issued by the President and regulations issued by the Civil Service 
Commission. This has given us flexibility over the years; so our 
system has changed, rather remarkably, I think, over the last 50 
years, even more remarkably, perhaps, in the last 20. Our own con- 
cepts of what was needed to accomplish this basic objective have 
been sharpened and changed . 

We also have another advantage, relating particularly to 
Tom’s comments about the relationship of the Civil Service System o 
the legislative and executive branches. In the Federal Government, 
the Civil Service System is clearly a creature of the Presidency. 

This was not quite so clear when Tom was with us, some years ago, 
not because of legislation, but because of attitudes that had grown 
up in terms of our relationships with the Congress and the dis- 
interest of the President, over the years, with the system. In the 
last 20 years, there has been an enormous shift in attitude. Now, 
we clearly have a situation where the Civil Service System, in all 
of its manifestations (not only in its recruiting and examining and 
selection program) is an arm of the Chief Executive and functions 
as an arm of the Chief Executive, not as an arm of the Congress. 

That doesn't mean we’re not subject to Congressional appropriation 
and Congressional control; of course, we are, but no more than any 
other executive branch agents. 

Let me turn from these differences to a brief characteriza- 
tion of a couple of principal points about the way in which we go 
about our business as it relates to the problem of teacher selection. 
We obviously have a very large system and we obviously are conducting 
examinations in the same way that the New York State and New Yoric 
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City systems are for wide varieties of positions. We are the 
single employer in the country. We have many different sub-systems, 
really.withL our total system, but I think the parts of the system 
that s^e most relevant to the subject we're talking aboutatthis 

Conference are the parts of the system whio ^f of 

mpnt pvamination, and selection processes that go to the iniing 01 

our entry level jobs in professionaXadministrative^dtectaical 

categories. So let me dismiss the stenographers, typists and the 
postal clerks, the sheet metal workers, the plunders and ^he ’welders, 
Md so on. let's talk about selection which is directed primarily 
those people who have prepared themselves through higher educa- 
tion We have basically two different ways of going about this job. 

I don't want to over-simplify, because it's JTtwo. ‘we 

two, but broadly speaking, we can ca egorzeou neople 

briug into the Federal Civil Service about 20,000 to 25,000 people 
every year at the college entry level, the bachelor s and mas 
decree level. We bring them in again to a wide variety of jobs, 
eluding, by the way, some teachers. This is not a large employmen 
category il the Federal Civil Service, but we have in our Bureau of 
Indian Affairs a very significant population of classroom eac ers^ 
at the elementary and some at the secondary level. bit 

experience in this area, and I'll touch on that in just a Uttle bit. 
Rut in this whole range of categories, basically we use two diff 
ent techniques. For about half of these jobs, we use f 1 ® x ^^g° n 
known as The Federal Service Entrance Examination. ° ut °* 
select between 10,000 and 13,000 people a year. We test between 
100,000 and 1.50,000 people each academic year, and this results 
this number of selections. This is a massive program, it's a P™ 
gram that involves general abilities testing, much along 
th“ have been talked about here and it involves an evaluation of 

education and experience. 

By the way, we have one restriction in our law that is 
troublesome, I should say, that most State and local Ourisdictions 
and industry do not have, and that is we may not require education, 
as a minimum requirement, except for professional an sc en 
positions. So, for many administrative types of positions , w 
to allow people to compete who do not have college degrees e 
though that might be the desirable way in which we would like to en- 
ter people. But for our professional jobs, such as teachers, e ^g - 
neersphysicists, economists, and so on, we do have positive educa- 
tion requirements. We provide for most occupations «^f "® ^® we 
identified as not in short supply and for which the test battery we 
use is appropriate, the Federal Service Entr^ceExamasthemetho 
to come in. We give everybody this same test. Weevalua 

»ation^ey a tove^ffli^Senwe r :electively a use°'this list otA- 

fo^job d sTe^ high 1 verbai n abilities*? t ' & verbal abilities 
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test score for selective use of the list. If there* s a need for 
someone with specific training, we*d like to bring someone into an 
agency where the job demands require some business administration or 
accounting background, not necessarily professional accounting. We 
can look for people with, say, 12 hours in accounting, as an indica- 
tion of interest. Agencies recruit heavily for a particular job. 

They identify people in whom they’re interested in the colleges and 
we have ways by which, in a perfectly legitimate use of merit sys- 
tem procedures, we generally can find ourselves able to reach the 
people that are really in demand because they will perform well and 
they have a record that is appropriate for the job. 

Now, in a large number of other categories, and this in- 
cludes primarily the engineering and scientific categories we have 
found, unlike A.T. and T., that we haven* t been able to get away 
with written testing. Here we have followed a procedure which is 
working extremely well. It’s not getting us all we need. We have 
not yet found anybody who gets all they need of top quality physi- 
cists, for example, or mathematicians, or engineers, but we’ve done 
pretty well with this. It’s a device by which our recruiter, who is 
an agency man, who is recruiting for his program, can recruit in the 
colleges where he can find the kind of talent he’s looking for, and 
as a licensed rater, who can act for the Civil Service Board of Ex- 
aminers, and rate the person on the spot, in terms of his faculty 
evaluations, his interview, and his college record, and can give this 
person a rating and offer him an appointment, if he's authorized by 
his agency to do so. Frequently, the delay here is more of a secur- 
ity delay, because so much of our needs are in the Defense Department 
and they decide they've got to check people out pretty thoroughly be- 
fore they make an offer of an appointment, so we have a delay there, 
but this is not a delay through the recruiting and examining system. 
This system works well in the shortage areas. It involves considera- 
tion of the background of the individual, considering everything they 
have done and everything we can find out from their college records. 
In most of these categories, we do have an educational requirement, 
so we're dealing with only college seniors, or graduate students. 

Relating this to teachers, we have a somewhat similar sys- 
tem in operation for the Bureau of Indian Affairs in its teacher re- 
cruitment program, in that we do have a device by which we give the 
teacher recruiters in the Bureau of Indian Affairs authority to make 
appointments . They work through our Inter-Agency Board in Albuquer- 
que and they operate throughout the country in the recruitment proc- 
ess. They have a tough job because just as you have a tough job 
filling jobs in East Harlem, we have a tough job filling jobs in the 
Hopi Reservation in New Mexico and Arizona. Let me tell you it is 
not easy out there and in fact many of the problems are very similar, 
in that there’s a cultural difference. Some problems are worse, in 
that there’s isolation that is extremely serious and extremely tough 



to live With, for these teachers, ^lefaway 

three or four teachers aBdavery^ ffi ^ snakes , 70 miles 

on a bumpy road i"*®®*®^,, L w hich has the filling station, and 
away from the nearest smal * n qn we have a tough job 

I'm not kidding, I've seen these = • while we haven't 

selling. But when we recruit ° n ^Tthan you have, 

been doing as well as we d t thi ^. y this yea r „ e 're going to 

we've been doing reasonably well, we uni £ metho ds. Ba- 

sasts * ssasr.- 

£‘£x,n ss :s 

afterwards. This is an inte g P time. We have, in 

killed us in the Civil Service sy ourselves to the fact that 

shortage categories, learne getting the records straight 

paper work really is just a matter of getting we 

later because everybody we can P°^ s ^ ly “^^Libly get them to 

would like to have, can havea 0 don't pay too much attention 

accept it and report for duty. So we dont pay^ ^ and we 

to the paper work, as I say, 1 ^ what P we consid er the primary 
don't allow that to ge ? ho that this procedure, or 

l° h > g -deas g tharhave been dis;ussed Lound here will be useful in 
further discussions during today and tomorrow. Thank you. 

Discussion 

Samuel Stretcher: The floor is open for discussion in the 

form of questions or comments. 

1 v, t t n+^v.. Mr Bereer raised the question whether it 
Ralph Walter: Mr. Berger r^s ^ ^ ^ ^ cQn _ 

is necessary S ?®° a a real problem with college preparation, be- 

cause there's increasing pressure for u ® p P ^ th disa dvantaged , 

. specialized fields like urban sohoo ^> ' 3 ihe ooint of view of selec- 
emotionally disturbed, and so on. involved in this prefer to 

tion of teachers, would you pe p trainine as possible, who would 

have someone with as broad a S®“® ® thev are on the job, or would you 
receive a specialized training M ^ n they are on the^ , 
prefer them to have some degree of speciaiizea uram b 

„ , c treicher • Anyone care to eomment? Would you want 

* ss. a. ^ » -».*•. — >. 

ized training in the pre-service period? 
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and some schools have developed a rather elaborate program, where 
they get special training in this field. Is this essential or is it 
better to have a teacher who is generally qualified and then gets 
some special experience and in-service training for urban schools. 

Samuel Streicher: Any of the people engaged in programs 

of teacher training care to comment? Please. 

George Redfern: I would like to try to answer the ques- 

tion. As a former personnel director, I would have welcomed special- 
ized training for the inner-city because the market research showed 
60 per cent or more of the vacancies to be filled were in that area, 
and I must say that most school systems have struggled to have their 
in-service programs adequate to re- train. There's another very prac- 
tical problem, that in the highly competitive market, it's academic 
sometimes to say to do it in the in-service method. If you don t, 
you can't hire them from your competitors anyhow. So, therefore, 
the elusiveness of many of these candidates and the competitive mar- 
ket make it an academic question as to whether you can have a real, 
honest-to-goodness in-service program, so to answer your question, I 
wish we could have some specialized training, as much as possible. 

It would facilitate - and I recognize your problem. 

Harry Gilbert: I'm Harry Gilbert, Pennsylvania State Uni- 

versity, Brooklyn, New York. I'd like to comment on a couple of 
seductive points that were made earlier. We're never going to get 
away from some value, judgments, despite all of our announced in- 
terest in hard research. But two points were made that I think are 
going to call for some value considerations: one was Mr. Berger's 

that we should not make tests public in the Board of Examiners, for 
example, which means you can re-use them and refine them, getting 
better and better tests. The second was that we should hire any col- 
lege graduate and then train him to become a teacher on the job. 

Now both of these sound very attractive on the surface, but I want to 
comment on some of their hidden meanings . 

In the first place, the Board of Examiners I know is unani- 
mous on this, takes the position that when you're hiring in the pub- 
lic service, you have a responsibility to tell anybody who is re- 
jected exactly why he is rejected. If you are going to keep your 
tests secret and refine them, etc., a person who is denied a. job, 
cannot look at that test, know exactly which items he has failed, 
and file an appeal, if he thinks he has been impr o per ly. selected. 

We take the value consideration that in the public service any ap- 
plicant has this right of seeing why he's rejected and a right of 
appeal. So I think it makes the job a great deal harder, not just a 
little harder, but I think it's a necessary part of examining in the 
public service. 
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Secondly, this business of hiring any college graduate and 
then making him a teacher comes to the heart of what I think is a 
real derogation of the teacher in our society. I work now for a real 
hard-boiled Dean in the College of Education at Penn State, Abraham 
Vander Meer. He used a figure of speech I'd like to present to you. 
He says, when I hire an architect to build my house, I'd like to have 
someone who is conversant with Camus, who knows the contributions 
that a Matisse has made to our aesthetic taste, knows all about some 
of the avant garde musicians, etc., but first of all he wants an ar- 
chitect who knows thermodynamics, and makes sure he puts into his 
house the properties that will make it resist the weather properly. 
Then he can have all the other characteristics. The same thing ap- 
plies to a teacher; if you think a teacher has to be one who knows 
children, knows how to present the subject matter which he knows 
thoroughly, this calls for a pretty highly developed integrated 
series of preparation. When you say let's take anybody,^ since we're 
in a shortage, you 1 re downgrading the need for this total prepara- 
tion, and in fact, you may be doing yourself an injustice. I think 
Doug Bray made that point. Don't sell yourself too short. In the 
long run, if you stick to the standards that you require, that in 
itself makes it attractive. So I'm going to keep yelling, even 
though one side of me is very much devoted to research, to the need, 
too, that we don't discard plain hard thinking and coming up with 
some values that we think are indispensable. 

Raymond Jacobson: On this issue of the publicity of tests, 
I just want to indicate that I agree with you that in a public sys- 
tem any applicant has the right to reconsideration and appeal, has 
the right to know why they are ineligible, if they are, or why they 
got a 72 instead of an 83. We in the Federal Civil Service, and I 
think this is true of most Civil Service jurisdictions throughout the 
country, state and local, do not feel that in order to meet that ob- 
ligation, it is necessary to publish our tests and to throw them into 
the kind of controversy that I think we get into that way. We've had 
no trouble with the Courts. We get an appellant who wants to find 
out how he did in the examination, the whole test rolled out in front 
of him. We tell them in which way they were weak, in which way they 
failed, we don't go over it question by question, and I think this 
public obligation can be met without regard to this kind of unneces- 
sary openness. 

Bernard Berger: Dr. Gilbert knows that up to July of this 

year we published most of our examinations. In fact, the Board of 
Examiners I think adopted our policy of publishing examinations some- 
time ago, but we have come to the realization that we're not doing 
ourselves a service as a selection agency in constantly pouring out 
our energies and writing new equivalent examinations which are not 
equivalent . We probably have as elaborate an appeals procedure as 
any jurisdiction has. It is quite likely that we have been involved 
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in more litigation than most other jurisdictions, so we are particu- 
larly cautious in our test and appeals procedure to avoid litigation 
as much as possible. We do provide, for example, every candidate 
with a blank numbered sheet where the candidate, on a multiple choice 
kind of examination, can list his answer as he goes along, and take 
that home with him. We publish the key answers, the tentative key 
answers, so that almost immediately a candidate can tell whether he 
has passed or not; secondly, we allow candidates to see the questions 
for which they were marked wrong — only those — and the candidate 
has an opportunity then and there to appeal or to come back again 
with some reference material to support his answer. We also know 
that in our test-wise population in New York City, candidates, if 
they feel that there is a true grievance that should be rectified, 
and if we have opposed their contentions, they go to the Courts. 

There are many lawyers in New York City who are willing to take these 
cases on some contingency basis . In this way the problem of appeal 
is pretty well handled. This is true, even more so, in the State 
Civil Service Department where a candidate can on occasion take an 
expert along with him to look at the question. So the appeal proce- 
dure is still there. 

Thomas Bransford: May I speak to this — I hate to leave 

to our most recent convert the job of proselytizing the Board of Ex- 
aminers on extending the useful life of good tests. 

We have been able for years to play one-upmanship with our 
colleagues in the New York City Personnel Department by saying that 
if the City Civil Service hit upon any good questions, they would im- 
mediately throw them away. We can’t use that jibe any more. The 
State’s examining program involves many candidates from New York City 
as well as upstate. We do have a system which, in some ways, is more 
rigid, more secure than the New York City system. It’s true that we 
do allow consultants to accompany candidates in reviewing some of our 
tests. Some of our tests have been exempted from any review. A 
clerical review of answer sheets and key answers without access to 
questions is generally allowed. In some examinations we allow can- 
didates who are willing to travel and make special arrangements to 
appear at special centers the week after a test has been held to re- 
view the tests and tentative answers under secure conditions to 
point out what they believe are errors. Some of our candidates are 
urging us to follow the New York City practice because they are not 
sure what answers they gave, and don’t know whether or not to object 
to the tentative key answer. We feel that one of the virtues of our 
system is that objections are not merely self-serving. Another vir- 
tue is that the exposure of the test questions is more limited in 
time and in place. In examinations where we allow review of both 
the test questions and the scored answer sheets, some candidates com- 
plain because we do not allow them to take tests home for extended 
research. I would like to join Mr. Berger in urging those whom Dr. 
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n . iw+ npf+ behind at the Board of Examiners not to be stampeded by 
Civil Ser^Se employee press away from constructive changes in 
^licy ^d p"cedure P fo^ safeguarding and reusing their best test 

questions . 

Harry Gilbert; Tom, I apologize; I meant to attack you, 

too . 

ppr’tfude Unseri It just seems to me that you can't have it 

Fither vou keep your tests completely secure, which means 
both ways. Either ^Keeprmr the \ ±Yid of procedure which 

Sfstate De^tment uses in administering scholarship examinations, 
hereby each booklet is numbered and is put into an enveiope. ane 
even the people administering the test may not see them or you allow 

some formVappeals procedure. As soon as ^ * P 
neals Drocedure, to that extent, security has been violated, une 
Lrson P may look at Item 10 and the other one at Item 50 and so on. 
E TZ put these together. I don't think you can straddle this 
issu^and* 1 pursue a middle course. You can deceive yourself that 
you're keeping them secure, but you re really not. 

Jules Kolodnv: I’d like to tie some looseends together^ 

-.r-s-jas 2 Ji: 

A third speaker, for example, spoke about the evaluation 

Of training prior to k ind C of e reouirements we ? have? Traditionally, 
Sf ^ses'^e^ldtn ^”em on.the American college 

^! eS thSe are ^rc^LeX^e^he^Se^igh^t esteem. 
How, 1 would wonder whether we c °^ p ^ 1 ^/ terms^f 

subject matter which they're supposed to teach. 

If you have a broader recruitment area and a re-evaluation 
of the kindof training you require, you might consider giving the 
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National Teacher Examinations as a Civil Service hurdle. You would 
have to consider what Dr. Streicher alluded to, namely, that the job 
must be made very attractive. We must make the position such that 
we want people to move into jobs which will have holding power. 

That's a matter which the superintendents and Boards of Education 
have to concern themselves with rather than selection officers. Once 
that is done, you may be able to have a very substantially high cut- 
off point on the National Teachers Examinations. Our particular com- 
munity may take people who score in the top decile, and that's our 
cut-off point because the conditions in our school system warrant our 
being this restrictive. On top of that, you must tie it in with the 
type of thing Dr. Rivlin indicated and some other speakers have al- 
luded to, namely, there should be an on-the-job training program. 

I'd like to think of it as an internship program. The medical pro- 
fession, for example, talks of an internship program^ we, on the 
other hand, go through what we think is a very careful selection 
process, and then we throw the beginning teachers into a full pro- 
gram, close the door, and he's all alone with a group of youngsters, 
never having done anything like this before. I should imagine that 
if we tie some of these loose ends together and make a complete 
"gestalt" in a broader recruitment area, using the National Teachers 
Examinations, with very substantially high cut-off points, and have 
a type of internship program where the teacher teaches a shorter day 
and has enough time to observe his colleagues and to talk to his col- 
leagues and then is evaluated at the end of a given period of time 
while he's still a probationer, you're more likely, rather than less 
likely, to get the quality personnel you're looking for. 

Harold Mitzel: There's still one loose end in Mr. Kolodny's 

remarks, and that is the modern teacher education curriculum. It is 
not true that the teacher education curriculum is loaded with methods 
courses, although that may have been true some years ago, but I'd in- 
vite him to take a look at a teacher education program and he would 
find a minimum of courses which he describes as "how to" courses. 

Milton Gold : I think we have to distinguish here between 

things that we are looking for. In terms of preparing teachers for 
the inner-city, we discovered at Hunter that the most meaningful 
things +nat we do are to provide students with first-hand experience 
in the situations, so that they're not afraid of it, that they can 
see if they can adapt to it, and that they can cope with the situa- 
tion that they have to face. 

We also know that, in looking for the prospective teacher, 
we're looking for a person who has a commitment to teaching and, in 
the case of New York City, who has a commitment to work in what may 
not be the most pleasant teaching situation, in terms of absence of 
pressure, tensions and so on. These things are quite at a variance 
with the kinds of things that we normally test for, and it isn't a 
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question of whether there's any test security or no test security, 
the question is how do you find this commitment, how do you provide 
a program of experience and training? Are we asking the examiners, 
now, to step out of the role of a narrow kind of examining and to 
partake in the problems of pre-service and in-service education? 

In line with what Mr. Kolodny mentioned a few moments ago, 
one of the interesting things we are doing is in cooperation with 
the Board of Education, working with some 2 $ teachers in Central 
Bronx schools, where they are given a reduced program, so that they 
may have some time to confer with each other and to work with people 
who are providing some kind of supervisory or academic assistance. 

On the one hand there are the cognitive things that Mr. Kolodny has 
mentioned, and nobody disparages the importance of it; on the other 
hand, the importance of survival in the school system often doesn t 
have much to do with whether a person is in the top decile of a test 
that's cognitive in nature. 



Raymond Jacobson: I just want to clarify something and to 

just add a point which may help with this area. The clarification 
is that I didn't want to imply that in our recruitment of teachers 
in the Indian schools that we rely exclusively' on. or require Nation- 
al Teacher Examinations. What we have done here is this (and it 
seems to me some of the comments we had this morning from, our indus- 
try people lead in this direction, or indicate that this is the 
direction that might be valuable for the profession as a whole),, 
what we've done is that we've identified what we're looking for in a 
teacher in an Indian school. We've identified this in a series of 
job elements. I don't have them in front of me, but there are some- 
thing like six or seven; one of them, of course, is knowledge ox 
educational methodology; another is ability to apply that methodol- 

ogy, and so on. 



We have used a variety of alternative measuring devices to 
satisfy, to consider these different requirements; for example, a 
National Teacher Examination Test score will have some value in say- 
ing what somebody knows about whatever that test measures, and we 
don't want to throw it away, we want to accept it if the person has 
taken it, but we don't require it, because they can demonstrate this 
knowledge in another way. They can demonstrate it by completion of 
work with high class standing and high teacher recommendations in 
education programs. So that what we've done is set up a series of 
the important elements, the key elements in th§ job, that are needed 
for iob success, and we've had a whole variety of measures against 
these. Now, it so happens that we haven't included in this variety 
of measures a written test that we administer. We will accept 
others. For example, we're interested in performance on the gradua e 
record exam, which they may or may not have taken. If they ve taken 
it, we want to know about it, it's part of the total record of the 



individual. We provide, in what we call an unassembled rating sched- 
ule, that is evaluation of experience and education, for a whole 
series of alternatives. 

The evaluations are made by the panel members who are them- 
selves supervisory teachers, principals, people in the education pro- 
gram of the Bureau of Indian Affairs, and they've all agreed on this 
ratine procedure. We've had an extensive training program with them, 
so we S understand each other on this. They operate as the Civil Serv- 
ice Agent, even though they are not Civil Service Commission employees, 
and they conduct the evaluation, and what they do is use this whole 
battery of information, to decide on an actual numerical score for 

each applicant. 

Murray Rockowitz: I’d just like to make this caution. We 

talk in terms of teacher selection, as if there is a teacher a 
we* re after. I’d like to cut across the comments of four or five 
people this morning. It is a problem of criterion. 1 * A 

basic, but the question is - criterion for what? Dr. Gold alluded to 
programs, teacher training for the inner-city, and I think that 
what’s necessary for us to determine is some kind of base which rep- 
resents for Mr. Jacobson entry into the public service. Then after 
vou’ve made the determination of this base, you try to wed the par- 
ticular qualifications of the entrant to the task involved. Now, 
somehow, research has to set up a number of criteria for the various 
kinds of demands made on teachers in various areas. I think n 
would agree, for example, almost a priori, that the qualities in the 
teacher necessary to teach a bright child are different from the 
qualities that are necessary to bring out the best in the under- 
privileged youngster who hasn’t been able to be identified as 
bright or not bright because we don’t have the instruments to do 
this. I think that this caution ought to apply to all our considera- 
tions here, and it should probably influence some of the research 
lines of attack that will be delineated tomorrow morning perhaps . 
That’s one of the reasons why I take exception to the National 
Teacher Examinations because, unless they can serve to determine the 
basic minimum from which we start to examine to meet all ^ la ” 
tions on the theme that are demanded by the schools in the urban 
centers in the world today, we’re going to put the imprimatur on an 
individual and then find that he just doesn’t work out in many de- 
manding situations in the urban centers of our time. 

Jay Greene: I want to say for Art Benson, whom I have 

heard talk on this subject very much, that he regards 
Teacher Examinations as a screening device to befollowedbyother 
types of screening, but I should also want to make clear that what we 
have been talking about in New York City and other places is actually 
a simple screening device. It is not test selection, where we say at 
the end of this procedure, we now have selected somebody who is a 

good teacher . 






cj+r.fln.yp'iv enough in New York City we are less prone to say 

«-» r u is.?rLs l rr^r 

not verbally, that ne nas +n be a successful member 

dren, who knows how to teach, wl J 0 is goingto be a^suoce isi ^ bg _ 

of the community, and a successful memb . t t ion 

lieve that you can go that far in that sort of situation. 

We believe that a teacher of English should have a hack- 

ground in the subject , 

screening instrument, similar these would-be teach- 

which has a national reputation to ?^. whe ^^ e "°^, many of 

^TesH ft^ ^ou'ffnd thatf rfbasis o! Ibout 1 to 

r^t/so^ get *. screening. 

Sdlyrhe r should e have r a , knowledge of fundamental methods of teach- 
ing° ™ough to walk into a classroom and proceed to teach. 

We also have an interview. In our: ^^fuf teacher? 1 " 
to judge whether the person is a w^m, i^jfdiildren or whether he's 
whether he's going to get along we simple screening device 

going to motivate them properly. No. it^ a simple s £ ^ 

based on this major premise^ that o ^ h ^ h ^f ab o at all that is 
municate withreasonabley y usi ng value judgments 

b thKr^ 

as ■-as.pr-sa: ss 

reasonable clarity, and so far a p 7 interview may be 

those fundamental things that may s ow P a iking about now, 
commented upon in the interview. What we re taltau g ^ de- 

really, is what some of you have been urging, a minimal ^ a B per . 

vice which, with value judgmen s, we c ^ this let the administra- 

son who is to step into a classr • That's all we have been do- 

tors take over in a probationary period. That s all we na 

ing, and that's all we're talking about. 

Samuel Stretcher: We have about four more minutes. Any 

further comments or questions? 

Bernard Berger: One of the things __ that was jerytoien^^ 

mentioned by Mr. Kolodny was 11 *?? a ' Ln ® ^° ” e ^ inl!: it's something that 
mentioning ^ ^ methods be- 

r a use\o evSion J noVetermination of appropriateness of 
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selection methods and recruitment of teachers is worth anything un- 
less you can hold onto or retain the teachers. This, of course, 
opens up a host of problems which go beyond an attractive salary. 

The provision of in-service training, the necessity for cooperation 
among the supervisors of teachers, in their dealing with a new teach- 
er, the entire work situation, the promise that the job holds in 
terms of satisfaction for the individual teacher, and so on, and 
very relevant problems. I don't think that you can possibly do any 
kind of evaluation, unless you speak in relation to turnover in the 
school and to retention of teachers. 

And by the way, in our giving the more general kind of en- 
trance examination for professional positions in New York City, im- 
plicit is a minimum of a one-year trainee program or an internship, 
where most of the time is devoted to formal training and a series of 
rotation of assignments, in order to first, allow the individual to 
determine if this is really his place; and second, to determine 
whether we want to keep that person. £11 of these positions carry a 
one-year term, prior to a permanent appointment, which also includes 
a probationary period. 

Fred Williams: I have one question I would like to ask Dr. 
Greene in the light of what he said. I'd like you to recall. Dr. 
Greene, the Chairman announced approximately 3-1/2 minutes ago that 
we only had four minutes, so you have 20 seconds to answer. What do 
you see is the particular advantage of our testing procedures at the 
entrance level? Over the National Testing Examinations, if at all? 

Jay Greene: Just three or four. Art says his is a screen- 

ing. It doesn't include any oral phases but we do have an oral. In 
some areas, as health education, art, and so on, we have a performance 
test as well, and then what was mentioned before. We have felt obli- 
gated to release our questions to people, so that they might learn 
where their errors were, and improve; whereas the National Teacher 
Examinations have security measures which prevent the releasing of the 
tests. These are the major reasons. 

Samuel Streicher: May the Chair be permitted a brief per- 

oration? We have placed on the table the many considerations that 
should be brought to bear on the question of the selection of educa- 
tional personnel. We've barely mentioned all of the important con- 
siderations. The knotty problems that grow out of these matters will 
be discussed later, I hope, in greater depth. Those who planned this 
program were really wiser than they knew. They brought on an inter- 
disciplinary interchange from business right through education. If I 
had the time, I would have told you of some of the experiences of 
mine in East Africa, where Standard Oil of New Jersey had a representa- 
tive who did recruiting. He said to me, "What can I do when the 
President of the Republic says, 'You must employ 25>,000 of my citizens 
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in your various gasoline station, or else*?” It was just a condi- 
tion, don’t you know, in the climate of employment that may bowl 
over all other considerations. 

We’ve run the gamut from the initial informal selection in 
the teacher training institution right through the last stage of dub- 
bing a person as permanently appointed and entitled to various tenure 
rights. Shelley said, ”We look before and aft, and pine for what is 
not." Later on, we’ll try to satisfy some of the pining by asking 
for what should be better selection processes, and what else we must 
do to find out about them. 

Paul Denn: Thank you, ladies and gentlemen. The meeting 

is adjourned. 
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Paul Denn: We had a full session this morning and we have 

another one this afternoon. This afternoon; s session is one which 
will focus directly on the problem of policies and procedures in t 
selection of teachers themselves. We have two panels, one of them 
dealing with the policies and procedures for teacher selection, the 
other with criteria and the problems of validating such procedures. 

The first of these will be chaired by my colleague, Arthur Klein. 

Arthur Klein: I think I express the point of view of most 

of the people whom I have spoken to when I say that thus far, our 
conference has been a very stimulating and provocative one. There 
are a number of questions that I should like to see answered by my 
own board during the next few months, and I’m sure everyone ^Isewho 
is engaged in personnel work or in teacher selection will take home 
with him some questions that he will want to deal with locally. 

In discussing teacher selection and the desirable proce- 
dures for selecting teachers, we are very fortunate today in having 
three men who have been intimately associated with these practices 
and procedures in three large cities. Associate Superintendent Wil- 
liam Brown of Los Angeles, George Redfern, in charge of personnel in 
Cincinnati for many years and now the Associate Secretary ofthe 
American Association of School Administrators, and Albert Schiff, who 
is Director of Personnel in the schools of Detroit . I know that our 
own Deputy Superintendent, Theodore Lang, who is with us now, will 
very much interested in hearing of the ways in which these gentlemen 
have met the problems that beset every large city in the country. 
We’ll begin with Associate Superintendent Brown. 

William Brown: Thank you, Mr. Chairman, and thanks to our 

New York hosts for including two of us from Los Angeles. We re g a 
to be here in the Catskills, have some real fresh air for a change, 
and attend this very worthwhile conference. I felt this morning s 
discussion, the presentations, and the questions and comments gave us 
very basic guidelines for the selection of teachers. I hope earnest- 
ly that the full report on this conference will be sent to us . It is 
a landmark in the study and development of ideas for 
of teachers. I hope some day Los Angeles can get federal financing 
for a similar conference. I’m very serious about that. I think i* 
would do us good in the West. 
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Wp nersonnel people live in terms of our situations. If I 
say some things that seem a little out of line with what is desirable 
i/some of your districts, forgive me. It may be just plain L 

Angeles . 

Our district persently has 800,000 pupils, with 25,000 new 

-p- • hi ™ . r 

municipalities and also some un- incorporated county territory. On a 

m «.sr.£b» * '« 

geles ^j?* ri ^*2oo r of S tS e 599°schools* 1 fflre U located I,1 in 0 disadvantaged 
know^hafonVee ^ ^ 

ways make access to these dis ri if * they 7 don't like Los Angeles, 

our examinations and get h g ^ placing the soor e from the 

find it's a very easy riae w ra pel Dire ctor and a short 
Los Angeles exam on the desk of the P briefly, that's the 

interview often result in a teaching job. Very bneliy, vna 

setting for my comments on our situation. 

we need 5 000 new teachers a year at the present time. 

The most serious ne^d results ^“^^^“XLcreasrf^Lrnover of 
demand due to compensatory e uca , , ^ eX panded in a 

our teachers recruitment P-gram^has had^ ^ ^ ^ 

major way and is continuing P ^ ldeas ^ pr0 grams, 

are adding to th ^^tfcloself with our teacher education institu- 
and to work especially close y vital role in this entire 

tions. We feel that they h ^ e ^tiSd tht ctments this 

process than we formerly realized, and I like Dro fes- 
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kinds of teaching assignments in our school systems. One or two ex- 
amples have given me great encouragement. The University of Califor- 
nia at Los Angeles now requires every student teacher to have half 
or all of his student teaching in a poverty area in our school sys- 
tem. This has been one of the most heartening things that has hap- 
pened to our system in a long time. It»s really helping our recruit- 
ment. I cite that merely as one example of the things that are tak- 
ing place increasingly between the Los Angeles City Schools and the 
twelve surrounding colleges and universities which are our primary* 
and one of our best, sources of new teachers. 

I»ve put on the wall a chart of the five phases of the re- 
cruitment and selection of career teachers. I hope these five phases 
reflect some desirable policies and procedures. 

The first phase is the recruitment program. There are the 
regular programs: 1) California campus recruitment— Our staff visits 

approximately 3£ colleges and universities in the state, two, three, 
or four times a year, depending upon need. 2) Out-of-state recruit- 
ment now includes visitations to over 200 colleges and universities. 
The special recruitment programs are used increasingly in terms of 
shortages and inner-city problems. One program involves internships. 
This is a program whereby a college or university, working closely 
with a district, invites mature adults (housewives, members of other 
professions, and the like) to return to college but not to the cam- 
pus. These are baccalaureate degree graduates who have had no 
teacher training. They are invited to prepare to be teachers by 
focusing teacher training, not only student teaching but courses in 
professional education, in a school in a disadvantaged area. We 
have 180 enrolled in one such program. 

Talent search is another program. This involves direct re- 
cruitment of baccalaureate graduates, which was necessary last summer 
to meet the elementary shortage. Operation Return is still another. 
There are always teachers in the community who can be drawn back into 
the profession. 

The newest recruitment program that we’ve added, and if we 
don’t add one every few months I’m surprised, is recruitment in 
Europe. We have found that there are approximately 700 teachers re- 
turning each year from the armed services dependents schools. Many 
of these have separated from the school systems in which they taught 
before they went to Europe. Through an arrangement with the Univer- 
sity of Southern California, the Los Angeles City Schools will inter- 
view in Europe for recruits for this program. We have gone to Hawaii 
Mr. Baldwin can tell you of his experiences there in seeking teachers 

Recruitment is a very difficult business. The competition 
is keen. We have to search out and proceed in every way we can to 
help our school system get superior teachers. That, briefly, is our 
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recruitment program. As I tried to emphasize earlier, that phase 
of the program will not succeed in its objectives unless there is a 
very strong alliance with the teacher education institutions. There 
are several forces at work trying to get our school system to take 
over its own teacher training. That is not our business. Our busi- 
ness is staffing the schools and teaching — not teacher training. At 
least one member of our State Board of Education would like to have 
Los Angeles relieve this shortage by general hiring of baccalaureate 
degree graduates, and training them to be teachers through the re- 
sources of the school system. We have to do some of this in critical 
emergencies, but we want to keep such hiring down as much' as possible. 

The second column of the chart outlines the first appraisal. 
This is the interview and contract offer by the recruiter. The re- 
cruiter may see the candidate in New York City, Chicago, anywhere in 
the U. S.j he may see the candidate on a campus in California. We go 
just as far as we can toward a contract offer at the time of inter- 
view. Personal qualifications are always appraised, and if all the 
college records required for appraisal purposes are available, an 
offer of a contract may be made at that time. On our local campuses, 
where the records are complete for our purpose, we not only offer 
the contracts, but the candidates, if they accept, talk immediately 
with our Assignment Administrators regarding the schools in Los An- 
geles where they may teach. Phase One, the recruitment, is the. 
reaching out, the searching for teachers ; and Phase Two is the inter- 
view, drawing them into the school system if they are evaluated as 
•>3ing qualified for one year of service, the initial employment. 

The next phase is the tryout year 5 that is the third column 
on the chart. This tryout year is extremely important, and I was 
very much interested in Dr. Bray’s response this morning, when he em- 
phasized the attrition in a tryout year. This is exactly what is 
happening in our district. We know that, with the kind of initial 
appraisal we have to make, there is going to be a definite percentage 
of the new teachers who can’t make it, who aren’t qualified for regu- 
lar service. Approximately ^ to 10 per cent of our new conditional 
or provisional teachers receive no better than average ratings, with 
the probability that a very substantial percentage of this group 
will not make it. They are encouraged to resign, and many of them 
do resign from the school system. Now I might emphasize that during 
this tryout year we see just as great an obligation on the part of 
our staff as on the part of the new teacher. The teacher must prove ^ 
himself 5 he must get ready for the formal examination. We must help 
him to know the Los Angeles school system, its philosophy, its cur- 
riculum, its teaching methods. This is extremely important, I feel, 
as a selection device, the two-way street during this conditional 
year of service. It’s a probationary year under State law. We can’t 
counteract that, but the condition for continuing on into a second 
and third year of probation is successful passing of the formal ex- 
amination. That is phase four, the most important one in maintain- 
ing the high standards of our school system. 
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All elementary teacher candidates must pass an English 
Usage Test to continue on to other phases of the examination. An 
objective test covering basic subject matter areas, methods, tech- 
niques and philosophy, has a weight of 30 per cent. An evaluation 
of training and experience by a committee has a weight of 1|0 per 
cent. A second committee evaluates personal characteristics. 

Now running all through this, there is a central strand of 
importance as far as we are able to ascertain from our correlation 
studies. The central strand basically starts with the strength of 
the recruitment program, whether the recruiter makes sound judgments 
in terms of qualifications to enter service, whether we unearth well- 
qualified candidates. If we can't, the whole process falls down. 

The judgment of the principal during the tryout year, as to whether 
the teacher is performing up to standard, as reflected in the per- 
formance report, becomes the most important factor in the formal 
examination as far as our studies have indicated. Now what we have 
here is a dual system of selection, an initial selection and then a 
formal screening during the tryout year. In the past, a number of 
years ago, we had just the formal examination system. It didn't 
produce enough people to staff our schools. Our operating divisions, 
elementary, secondary and college, strongly favor the dual system. 
They want the Personnel Division to get good people, after initial 
screening, into the school system. They want to try these people 
out, help them get started, and be perhaps the most important factor 
in determining whether they stay on and become career teachers. So 
this is the heart of the selection program. I think it is desirable 
for our system. Whether it is good for any other districts, I have 
no idea. This double screening system makes a heavier load in proc- 
essing, but there are some clear indications that it is working well 
for our school system. The fifth and final phase is the second and 
third year of probationary service which provides the final apprais- 
al before the teacher enters upon a career job. Very few teachers 
are terminated during these two years of service. 

Briefly, that's the way we select in Los Angeles, I've 
learned some things this morning that have already given me clues 
as to where this procedure can be strengthened. I won't go into 
those now. I found particularly helpful some excellent points on 
teacher education and the great importance of a closer working re- 
lationship in terms of recruiting early in the college career, in 
the freshman, sophomore and junior years, and getting better records 
in the college as a sounder basis for our recruiters to appraise 
these young pelple before we decide whether they should serve with 
us . 

We have a number of problems. One is the necessity for 
the dual screening. Another involves the credential problems which 
the state imposes on us. Los Angeles requires 5 years of preparation 
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for all teachers, elementary and secondary, and trying to keep a 
five-year standard in the present period of critical shortage is 
extremely difficult. We have considered requesting the State to 
give us our own credential structure. 

We have problems in training provisional teachers. Four 
hundred were hired in mid-August to relieve an elementary shortage. 

We had two weeks in which to give them intensive training in our 
schools — that wasn't enough. We have master teachers helping them, 
but some of them are not succeeding. We hope the majority of th*m 
will make the grade. 

A third problem is that of competition with our surround- 
ing districts. How a large school system can hold onto the good new 
teachers it gets, I don't know. Our turnover rate for new teachers 
is fairly high. We know that salary is an important factor both in 
recruiting holding teachers. We believe our district has to 
have one of the highest teacher's salaries in California, just as 
Dhase of a good selection program. Our minimum now, with a bacca- 
laureate degree and no experience, is $6,220 for the first year. We 
feel that salary must be at or near the top in a state for a large 
city, which has a definite part of its district in poverty areas, to 
retain high quality teachers. 

There are some favorable factors in the picture at the 
present time. College enrollments are increasing in our area and 
we are encouraged by this . We believe in building up e pres ige 
of teaching, doing the right kind of recruitment job and getting out 
into our colleges more. I find, as a recruiter for the Los Angeles 
City Schools, that one of the first places that I have to go in a 
college or university is the president's office. The presi en mus 
be in on the organizing of the college to help teacher education 
prepare young people for the jobs we have available. can 
placed upon the Dean of Education alone. He needs the support of 
the president, and all the faculty. 

The innovations in teacher training are very encouraging. 

I believe this is a vital part of desirable selection. . Many new and 
promising programs are evolving in our colleges and universities. 

We should plan for the return of former teachers to serv- 
ice. 1 think the only way we will meet the problem of good staffing 
is to think of every young woman who enters elementary teaching 
LOS Angeles as a possible returnee. We know she *°“ '* J s *f 
in most cases, probably two to six years; then she will be out f 
to ten years to raise a family; and then she may return to service. 
When these women return, they are typically stronger teachers than 

when they left us. 
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The compensatory education programs, and we in California 
fortunately have several State programs that are helping out, are 
beginning to make teaching in the inner-city more attractive, by 
lowered norms, increased remedial instruction, and specialized as- 
sistance to the pupils in various fields. Something is happening 
here in the United States in our large cities that I believe is go- 
ing to be an important factor in drawing strong teachers to the 
central city and encouraging them to stay there. I go into the 
classroom in school after sohool, and see a te*cher with a small 
group of children around her learning the elements of English. This 
is heartening and our teachers are enjoying this kind of service. 

We have had excellent community response to recruitment 
appeals in the face of shortages. I believe that our communities 
can work much more closely with us than we have asked them to do in 
the past. When we reached the middle of August this past summer and 
found that we were 800 teachers short, we had already gone through 
all of our existing recruitment programs. There were no more teach- 
ers available. We went to television; we went to radio; we went to 
the newspapers. Nine hundred applications came in, and 600 were 
employed. This was one of the most heartening things that has 
happened in our school system. 

Mr. Chairman, I think that very briefly tells you what we 
have been trying to do and the direction in which we're working. I 
haven't gone into details on the parts of our examination program. 

I believe we heard some very important things this morning on that. 

I want to repeat that this has been a most worthwhile ex- 
perience for me, and I want to thank all of you for the opportunity 
to share with you in this conference. Thank you very much. 

Arthur Klein: Thank you, Dr. Brown. I'm sure that there 

will be some questions later after we've had an opportunity to hear 
from the two speakers who will follow you. Dr. Redfern, now in 
Washington, formerly in Cincinnati, will tell us about his personnel 
experiences in the selection of teachers in Cincinnati. 

George Redfern: Thank you, Dr. Klein. Members of the 

panel and colleagues of the conference. May I say, at the outset, 
that even though I am now associated with the American Association 
of School Administrators, I am still interested in and will be work- 
ing with large city problems. I am certain that one of the prime 
large city problems is likely to continue to be that of teacher se- 
lection. The only difference in my present situation is that I'm 
not obliged to worry about meeting that September teacher employment 
deadline when school opens. I can also look at the teacher selec- 
tion problem from a somewhat more academic point of view. 
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that references are provided by those who have had direct supervisory 
contact with the applicant and presumably are in a position to make 
relevant judgments as to the competency of the candidate. 

The third selection criterion is the oral interview. Nor- 
jjio^ly it is a group interview and conducted wherever possible at 
Board* of Education headquarters. A gratifying number of applicants 
are able to come to Cincinnati for the oral interview even though ex- 
pense money cannot be provided. About £0 per cent of all P^P 1 ® e ™“ 
ployed come from outside Ohio, which represents a considerable loa 
of interviews. Most of these out-of-town candidates are interviewed 
on Saturday by interviewing teams organized by the personnel division. 

The three selection criteria are weighted. Academic record 
counts 30 per cent; professional references, 20 per cent; and the 
oral interview, ^0 per cent. 

As I understand it, my position paper has not been put in 
your hands, but I’m going to risk over-simplification rather than 
bore you by reading the paper which gives more detail regarding the 
Cincinnati selection process. 

The selection procedures used in Cincinnati are not unique— 
they aren’t unusual. However, I’d like to make some observations 
about the type of teacher selection which they represent. 

1. Selection has to be made within a framework of reality. . As 
Dr. Brown has said, that reality is severe in large cities. So it 

is in Cincinnati where we have only a fraction of the demand that Los 

Angeles has. 

2. Another reality is the negative image of teaching in the 
large cities. Despite persistent effort to change that image in the 
minds of candidates, they still find suburban schools more attractive. 
In Cincinnati, for example, the suburban schools can and do attract . 
over half of all the graduates of the University of Cincinnati. 
Formerly, more than two- thirds of its graduates were employed in the 
Cincinnati Public Schools. 

3 . Even though universities in or near large cities may assign 
students to do their student teaching in the inner-city schools, 
this does not guarantee that the graduate will seek employment in 
the city. Our experience in Cincinnati confirms this observation. 

U. Another reality is the matter of high demand and low supply 
especially in critical teaching areas. You know the familiar ones. 

A fifth reality is the lack of good position specifications 
which are needed in recruitment and especially in placement. This 
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is a problem within the capacity of the school system to correct, how- 
ever. There may be many specifications which each vacancy requires. 

If principals fail to specify these requirements, recruitment and 
selection is made more difficult. 

6 . Another very significant reality is the wish of the candi- 
date to know where his placement will be at the time of employment. 
Most large cities have not found an effective way to assure a spe- 
cific placement at time of employment, especially during the spring 
recruitment season. This is a handicap of serious proportions in 
large city teacher selection. 

7 . An additional reality is that there are two general classi- 
fications of applicant. One is the interim teacher; the other, the 
career teacher. The former has short range interests; the latter, a 
career expectancy. Much that has been said this morning by our busi- 
ness and industry panelists has been more relevant for the career 
than the interim teacher. 

8 . An eighth reality concerns money. In Cincinnati, we have 
a salary policy that attempts to keep teacher salaries competitive. 
Salaries are fixed at the 7£th percentile of the hi largest cities. 
While this would seem to establish a favorable competitive position, 
it cannot overcome the many advantages— monetary and non-monetary— 
which competitive suburban communities possess . 

9 . A final reality is the rather unique requirement that em- 
ployment in school systems peaks sharply at one time during the 
year; namely, in September with the opening of school. Many other 
professions are able to smoothe out their employment needs. My con- 
tention is that since all school systems have this peak September 
employment problem, more mistakes are made in teacher selection than 
if employment needs could be more equally distributed throughout the 
school year. 

In the time that remains, I wish to make some more obser- 
vations about the non-examination type of selection procedures. 

After listening to Dr. Mitzel and Dr. Medley, I admit that I don't 
feel very comfortable about the adequacy of the Cincinnati teacher 
selection criteria. Admitting the weaknesses of the criteria, I can 
only say that we are obliged to live with them for the present at 
least. 



I am convinced, however, that the criteria can be refined 
and should be. The following steps may be useful in this refinement 
process: 



1. Better position requirements mus* - • prepared by principals 
describing the expectations of the inner-city teacher. With a 
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clearer understanding of what is expected of the teacher, selection 
may be made more effective. 

2. Each selection criterion should be carefully evaluated. An 
attempt might be made to correlate each with a success criterion 
which possibly could be ascertained from the appraisal records of 
the teacher during the probationary period. If the selection cri- 
terion correlates favorably with the teaching success criterion, a 
greater degree of confidence can be manifested in its effectiveness. 

/ 

3. Professional references might be M field tested” by a sam- 
pling process. For example, a random sample of written references 
might be verified by telephone calls to the individuals who made the 
references • The purpose would be to see if a conversation would 
yield additional or different information than was indicated on the 
original reference form. 

I 4 .. The oral interview could be scrutinized. Types of inter- 
views might be compared , i . e . , variations might be made in the com- 
position of interviewing teams to see if one combination appeared 
more effective than another. 

£. The priority given the various eligibility groups might be 
checked for validity. For example, scores achieved in the highest . 
priority grouping might be correlated with the teaching success cri- 
terion achieved during the probationary period. It might well be 
established that high priority group candidates do not consistently 
turn out to be high teaching success teachers. If, on the other . 
hand, they do, it would tend to verify the logic of appointing first 
from the top eligibility group rather than the next highest group. 

There are probably other ways to evaluate the selection 
criteria. The point is that refinements probably can and should be 
made in the procedures. 

In conclusion, I sincerely hope that this conference may 
advance the cause of better selection of teachers for the large 
cities. We need better ways to exchange and disseminate information. 
We need more dialogue between school personnel administrators and 
members of boards of examiners. We likewise should have closer con- 
tacts with those, at the university level, who are preparing teach- 
ers. We need the help of those in the universities who have ex- 
pertise in research design who can assist school personnel people 
and board of examiners personnel do a more effective job of evalu- 
ating selection techniques. 

I hope that out of this conference may come a better 
mechanism for exchange and dissemination of teacher selection data 
and information. I also hope that if the American Association of 
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School Administrators, as an organization, can be useful in helping 
large city personnel administrators fulfill their roles better that 
it may have the opportunity of doing so. Thank you very much. 

Arthur Kleins Our third speaker. Dr. Albert Schiff of 

Detroit . 



Albert Schiff: Mr. Chairman, colleagues - no city school 

system, worth its salt, wants to lower teacher standards, but the 
needs of the community are immediate problems and cannot wait for 
long range plans. Teacher selection has always been a calculated 
risk, but we can minimize this risk so that even the probationary 
period of which we f ve spoken in some of our sessions is a period of 
strengthening the teacher and not another screening device. We all 
know too well the incalculable harm an incompetent primary teacher 
can do during his first year of teaching to a group of youngsters 
just learning to read. This is unforgivable. 

It has been ten years since I've entered the personnel 
field. About 600 to 700 new teachers were all that the city of De- 
troit needed to meet its needs in 1956. We now search for 2000 
teachers every year, to help meet the needs of the growing community, 
to lower pupil- teacher ratio, to keep both the Federal programs as 
well as our local school programs operating and to offer all the 
other services that we've always said we needed. Even with funds 
available, we are short of personnel, and 2000 teachers are getting 
rougher and rougher to find. 

What we have to do - what we must do - is re-examine the 
supply lines of teachers. Present training facilities are numeri- 
cally and qualitatively inadequate to meet the needs of education 
today. They must be reorganized, they must be reformed, they must 
be augmented by new types of programs, and in many cases new types 
of institutions. We must make a nation-wide analysis of teacher 
training and the utilization of teachers, so that every level of 
education, elementary, secondary and college, can be assured of a 
steady flow of high quality instructors. This is the crux of the 
problem that big city schools face. 

For the most part, large city school systems have set up 
screening devices and techniques to weed out the less qualified and 
the less desirable teacher candidates who have already been gradu- 
ated from teacher training institutions. Boards of Examiners and 
other personnel officials, and Detroit is included, set up programs 
of testing and interviewing and examination of credentials, at great 
expenditure of funds (I just heard yesterday that the Board of Ex- 
aminers in New York spends $1,500,000 on this) to insure that their 
school systems do not employ teachers who can give less than quality 
education to their students. This is deplorable especially at a 



170 



time when staffs of this nature could be devoting their time and 
their energy in working out newer and better ways of selecting teach- 
ers for the present and for the future. No profession has such a 
rate of turnover and mortality as the teaching profession. It thus 
becomes apparent to many of us that we have to find the techniques 
of selection that will identify the candidate who has a commitment 
to teaching and will subsequently remain in the field for a life- 
time career. 

Selection procedures as they pertain to personnel adminis- 
trators begin with the college senior and, although our selection 
procedures in Detroit are perhaps as adequate as most large city 
school systems, I question the continuing need for teacher selection 
by personnel administrators of school systems when the selective 
process for future teachers should begin much earlier than the col- 
lege student who is approaching graduation. In Detroit we're often 
hurled this challenge, "Why can't we compete with the suburbs in 
terms of quality education, class size, salary, etc.?" Let's not 
fool ourselves— the sooner we face this problem realistically and 
indicate to all concerned that large city school systems, such as 
Detroit, New York, Chicago* and others are not suburban areas, that 
our teachers must reflect the ideals and the characteristics which 
are necessary for effective teaching in a large industrial and cos- 
mopolitan city. Then and only then, can we come to grips with cer- 
tain basic procedures to be followed in the selection of teachers 
for the future. 

In Detroit, over the past few years, the Personnel Divi- 
sion has sponsored clubs for future teachers, and I don't want to 
minimize this. In 19!?7 at the beginning of our efforts, we had 12 
high school clubs serving 300 members. We now have almost ^000 
pupils participating in the l6l school units. An average of 2^> to 30 
members is the usual size of these groups, although some clubs range 
as few, at the last count, from 8 members in one school to 130 in 
another. Even in our society, as mobile as it is in terms of changes 
teachers may make during their lifetime, the greatest source of 
future teachers for any large city rests with the high school stu- 
dents in the classrooms of that city. I hate to use an old cliche, 
but it's so true, that the "teachers of tomorrow are in our high 
schools today," and if you go too far away in recruiting teachers - 
at least we find this to be true - the less chance we have to get 
teachers to come to Detroit. 

There's not much solace in finding that school systems are 
competing for the pitifully few highly qualified, competent teachers 
available. I am also weary of these wonderful, beautiful recruit- 
ment brochures of how to attract teachers to our respective commu- 
nities. These techniques may be very well for business, but I often 
wonder if it is desirable for educators. 
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The selection of prospective teachers should be continued 
through the college and university level, beginning at the freshman 
year. School systems should provide a co-ordinator to work with 
teacher training colleges in their areas. The aim should be to help 
the prospective teacher and suggest shortage areas. Isn‘t it a para- 
dox that we still have a surplus of social studies teachers and be-, 
cause of that surplus we’re having them teach math and other areas in 
which we have critical shortages? Let us provide child study oppor- 
tunities through classroom visitations, teacher conferences, get them 
jobs as camp counselors, secure employment in year-round youth 
groups, aid them in materials for class papers. In brief, we have to 
extend the services provided by teacher education. and education. re- 
search divisions to the end that greater cooperation can be. achieved 
between the school system and the teacher training institution. Bill 
Brown, George Redfern, and I all agree that this close cooperation 
with teacher training institutions is important and imperative, and 
the internship program can be an extraordinarily successful training 
device leading to a real functional partnership between teacher edu- 
cation institutions and public school systems. Let’s look favorably 
upon the use of Federal funds to help us in a search for quality 
teachers for the future. Dr. Brownell, former Superintendent of . 
Schools in Detroit, who is attending this conference, gave us permis- 
sion to modify our teacher standards last year. We had an experi- 
mental group of seventy candidates enroll at Wayne State University 
who were post-degree candidates with little or no college credit in 
education. They worked with people out in the field under careful 
supervision and along with staff of the teacher training institution. 
We still have over forty of these people enrolled now and this is 
their second semester. While this may appear to be a small number, 
we’ve gained forty teachers by means which proved successful. There 
must be other programs that are equally valuable. 

Special needs of a metropolitan area, and this of course I 
do not need to call to your attention, demand racially integrated 
school staffs and training of prospective teachers to meet the needs 
of deprived or disadvantaged youth. We do call upon teacher gain- 
ing institutions not to recommend candidates for teaching about whom 
there are any questions of general intelligence or competence, per- 
sonality, or other factors which preclude success in teaching. 

School oersonnel administrators have a right to assume that these 
candidates are ready to "tackle" the rigors of the classroom if they 
have been or are given the aid of adequate supervisory and administra- 
tive staff. I believe large city school systems will need to modify 
rigid teacher recruitment and selection procedures to alleviate the 
teacher shortage by adoption of temporary stop gap measures while 
still attacking the problem of effective long range problems. 

Dr. Brownell laid the groundwork this past. year for work- 
ing out plans with the teacher training institution in our area. 
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Five Task Forces are at work at the present time. Task Force I is 
devoting its time to the development of teacher internship programs 
for undergraduates and post-degree students— and the development of 
specially designed post-degree and/or M.A.T. programs in teacher 
education (campus and off-campus). Task Force IX is working in the 
area of expansion and modification of current programs for the prepa- 
ration of teachers for inner-city schools. Task Force III is 
in the development of specially designed programs for the preparation 
of teacher assistants and para-professional aides (campus and off- 
campus). Task Force IV is currently developing policies and proce- 
dures for the recruitment of undergraduate and post-degree students 
for teacher education. Task Force V is devoting its time to the 
organization of research teams for conducting follow-up studies, ac 
tion research, evaluation and assessment of teacher education 

programs . 

It is the plan of these Task Forces to have some recom- 
mendations in operation for the February, 1W, school semester. 

Thank you very much. 

Discussion 

Arthur Klein: When I read the position papers that were 

submitted by the three gentlemen from whom you've Just heard, I was 
delighted with the forthright and realistic approach that they had 
to the great problems confronting personnel administrators in the 
large cities. I was amused, of course, when I heard one of the 
speakers repeat the question that all of us are familiar with l 
laree cities* "Why can't you compete with the suburbs. Well, of 
course yo^eoplT can't supply the answer in Cincinnati, for example, 
because you haven't an organized testing program. In New York City, 
the people who ask the question are always sure of the answer: I 

your darn examinations that keep them out." 

I know that you have questions that you'd like to ask 
these gentlemen. One or two comments first. You may recall that 
Harrv Rivlin pointed out the importance, which has been underscore 
by some of our speakers this afternoon, of the selection process 
that begins in the colleges . Then there is the. examination proper 
by the selection agency and thereafter there is the probationary 
period. There is a continuing selection process over a period 
years, but I was glad to hear Dr. Schiff voice very forcefully the 
thought that one can't subject children to untrained or undesirable 
teachers, merely because we can fall back ontheprobatronary perrod 
for the elimination of those teachers. Dr. Rivlm left with Fred 
Williams a message to be read to us. Fred. 

Fred Williams: Thank you Mr. Chairman. Dr. Rivlin ap- 

parently felt that out of the conversation he had last night there 
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may have been some misinterpretations 
clarify his point. 1 wasn't present, 
fusion was, but I'm sure this will be 
I'll read this as he gave it to me. 



and wanted 
so I'm not 
of benefit 



the opportunity to 
sure what the c on- 
to all who are here. 



» The keepers of the gate are key people in any school sys- 
tem. I do not question the need for a selection procedure, but I 
hope we can find better ways of selection. I think research money 
will be invested better in improving our ways of evaluating perform- 
ance on the job than by concentrating on the improvement of the ex- 
amination procedures for determining entrants into the system. 4 ere 
must be selection for initial appointments, but this is only a screen 
ing test. Those who are passed should be 'accepted as promising 
risks' and should be evaluated during their period of probationary 
service before being granted tenure. Provision for . evaluating prior 
experience is important for two reasons: one, previous experience 

may be a useful part of the screening process 5 and two, such evalua- 
tion can be part of a recruitment process . 



"Why shouldn't the Board of Education raid other school 
systems by inviting unusually successful teachers and principals to 
join New York's school system? Can the Board of Examiners find the 
time and personnel resources to focus on the evaluation of perform- 
ance, if it relieves itself of the time-consuming chore of. conduct- 
ing, administering, and rating written examinations by availing 
itself of such prefessional test agencies as E.T.S? How can we ge 
good experienced teachers and principals from other school systems 
and yet be fair, objective and honest? For example, can we give the 
examinations privately to those who have been evaluated in their job 
in their own home settings? The Board of Examiners must continue to 
be the keeper of the gate, selecting impartially. and honestly, re- 
gardless of race, creed or political power, but it also. has a re- 
sponsibility for evaluating those who get through the first set of 

gates . " 



Arthur Klein: And now questions addressed to our three 

speakers . 

Ralph Walter: Dr. Schiff stressed the need for the col- 

leges to cooperate with the city school systems. As far as the ex- 
perience of the students is concerned, I think there is another. side 
to that coin. We've just gone through this experience: in. junior 

year, we have 1000 juniors who go out and spend part of their time 
in city schools and part of their time in suburban schools. We 
found great cooperation in the suburban schools, but we could not. 
place our youngsters in the city schools. Two of the largest cities 
in the state turned us down on this, and I think if something could 
be done to induce them to accept, it would be helpful. 



Albert Schiff: Someone here said that's hard to believe, 

I don't think that that's going to happen in New York bity, an D £‘ 

Lang has just said he'll take them in New York. So. you tell the 
nighted people in New Jersey that the Hudson River is fordable. 

Jules Kolodny: I'd like to direct this question to Dr. 

Brown. You referred to visits being made in- state to several hun- 
drpd colleges and out-of-state as well, to several hundred colleges. 
You've talfed^f visits to Hawaii; you've talked of visits in connec- 
tion with the Army schools, returnees, recruitment ara° n g e * 
laureate graduates who were in other professions, etc. May I a 
these two questions: how many people are assigned to is as , an 

what is the size of the budget appropriation? 

William Brown: In answer to your question, there are two 
forces involved. There is a force of about 1$ professional recruiters 
who work year-round. In addition to that, many school principals ar 
relieved of their assignments for one, two, three, or four weeks de- 
pending upon the length of the trip. Supervisors from the operating 
divisions are used. So, the Personnel Division budget for recruit- 
meS would include the salaries of the 1? regular staff members with 
an average salary of approximately $1^,000, and. the salaries of t 
principals and the supervisors. Wayland (Baldwin), you estima 
once how much it cost the district to use the school personnel. 

Whs/t would that cons to? 

Wayland Baldwin: I don't have those figures at the present 

time In our whole selection procedure, including examination com- 
Vt^es where we make generous use of our school principals, .super- 
visors vice principals, and teachers, and also in recruitment 
effort, we use about a thousand people a year. I would estimate that 
this would average about three or four days for each one, on the aver- 
age. This is a district-wide enterprise and runs into several 
hundred thousand dollars, but I can't give you any exact figures. 

Charles Cogen: I would like to make three comments. Two 

of them are more or less of a negative nature, one of a positive 

nature . 

A good deal has been said, and I think there have been some 
fine suggestions submitted by these speakers, about the role of the 
teacher!' college and cooperating with the school systems and so on 
Now I think one consideration that we have to bear ? 

adequacy of many teachers' colleges. We hear an awful lot. about, 
and^e see— I've seen in my experience— students who come into the 
classroom who had never heard of a lesson plan. and other realistic 
things in our teaching which are a little foreign to the subjec ma - 
ter of many faculties of teachers' colleges who teach much, much 

too theoretically . 
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Secondly* another negative factor is that we have to be 
concerned about the supervisors in this whole role of teacher evalua- 
tion and screening. Now this has hardly been mentioned at all in any 
of the discussion that we've had up to this time; namely, the fact 
that supervisors frequently are very incompetent to do the job oi 
assisting the teacher or may have personalities that react unfavor- 
ably or have a laissez-faire attitude. There are all kinds of atti- 
tudes on the parts of supervisors which militate against plowing in 
to the teacher that which is needed to bring forth a good product, 
and this is something that I think bears very much consideration. 

How do you take account of that in comparing one candidate with 
another when they've had different kinds of supervision? 

Finally, a more positive note: the question has recurred 

over and over again, not only at this session but at other sessions, 
of the problem of the inner-city and how we can recruit and re a 

teachers for these difficult schools. This is probably one of the 

crucial problems of the day in education. I think, in that con ™;C" 
tion, we ought to mention— and I don't see how we can avoid mention- 
ing— the fact that we need to pour tremendous resources into the 
schools in order to make the job attractive to the teachers and to 

hold them. In that connection I want to refer only briefly to the 

More Effective Schools program in New York City. I have visited a 
couple of them and I have seen them firsthand, and I ve heard much 
more about them second- and third-hand. I won't go into detail 
since I think most of you are familiar with the situation there— 
the saturation of services, and so on. I understand that Los Ange- 
les sent an investigator to New York City to see what is happening 
in the "more effective schools" and came back with extremely glowing 
reports, and said that this is the thing that will solve our dis- 
advantaged school problem; and yet, to my knowledge* Los Angeles has 
not introduced that program. 

Theodore Lang: I haven't had the privilege of sitting with 

you this morning and yesterday evening, so I do not have the benefit 
of your earlier discussions. It is good to be with you. It ±s 
comforting to know that other large metropolitan areas have problems 
almost identical to ours. As each of you spoke this afternoon, °« 6 
of us from New York nodded our heads in agreement. In the spirit 
learning, I have questions to direct to a number of the panelists who 

spoke here today. 

But, before I ask those questions I would like to recommend, 
because no one spoke of it, the technique of manpower projection to 
determine personnel needs for September at an early enough date 
that there can be better preparation than is the case when aschool 
system first knows of its critical shortage in August. We made --- a 
September 1966 projections last February. As a result, we were abl- 
to anticipate a teacher shortage of approximately 2,000. (.ihis 
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deficit steins from the same factors that account for your shortages, 
primarily, the availability of Federal and state funds for expanding 
compensatory education.) Thereupon, we trained approximately 2,000 
college graduates during the summer by enlisting the assistance of 
one of our best teacher training colleges and conducting an inten- 
sive teacher training program. The program extended from six to 
seven weeks, giving the participants, who had no previous pedagogic 
preparation, six or eight credits of professional background for 
secondary school and elementary school, respectively, so that they 
could be brought into our system as teachers in September. Of 
course, they were not adequately trained even with that depee of 
preparation. However, it is a much better approach than first recog- 
nizing your problem in August or September, and then having to re- 
cruit and place in the classroom baccalaureate degree holders without 
any professional grounding. So, I would strongly urge you to de- 
velop the technique of projecting your manpower needs to determine 
the probable shortages. This should be simpler in. your systems than 
in our organization because we have a huge group of substitute teach- 
ers (about 30 per cent of the total) and our method of assigning 
these substitutes, in distinction to the appointment procedure that 
applies to regular teachers, is such that we are not able to compile 
accurate data on the inflow and outgo of substitutes. 

Now, on the specific questions, in order to better under- 
stand each of your presentations - To Dr. Brown: I would be inter- 

ested in knowing, of all the contracts that you make with teachers, 
what percentage result from contract offers made at campus inter- 
views? And my second question to you concerns the anticipated 
failure rate: What is the proportion of teachers who fail your 

formal examination after having completed one year of initial teach- 
ing with satisfactory performance? 

William Brown: Before I answer your question, I would 

like to underscore the importance of manpower projections. We feel, 
from a recruitment standpoint, that if we cannot project well ahead, 
we cannot do the best recruitment job. Also, we have a serious fore- 
casting difficulty at the present time because we do not know for 
certain that a new State Law in California will be in operation in 
our district for February. This law would increase our needs by 330 
additional elementary teachers. There are some serious obstacles in 
applying to the State Board of Education and meeting the provisions 
of the law. I concur strongly in the importance of projecting ahead 
insofar as that’s feasible. 

Theodore Lang: The second question was expected failure - 

of those who have the one year of initial teaching experience, who 
take your formal examination, but limiting it to those who had satis, 
factory performance reports, what percentage fail? 

William Brown: Those who were strong or outstanding? 
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Theodore Lang: No, those who were satisfactory. 

William Brown: Approximately V? per cent. Dr. Lang, would 

be about as close as I could come on that. 

Theodore Lang: Thank you. The question I have for Dr. 

Redfern is on the oral interviews: You say that approximately 5 P 

cfnt ^e the oral interviews. What happens to those who cannot come 
in to headquarters for the interview, which is a weighted part of 
your examination? 

George Redfern: The campus interviews, in practically all 

instances, are made by a team and the oral interviewthenis. on- 
ducted on the campus by the team. Now, I would say that about 85 
per cent of the people at the campus level, would have a two-team 
interview, but there are some instances of only one interview. 

Theodore Lang: Do you make many contract offers at the 

campus? 

George Redfern: We make no contract offers at campus, 

even though we need them badly. We will not give anybody a contract 
at the time of the interview, until he processes the application. 

Theodore Lang: The question to Dr. Schiff: Dr.Schiff, 

one of our serious problems is the nature of the initial _ assignment 
of the teacher. I know it varies from school to school “New Tor 
depending upon the principal. However, we hear may stories “ Head- 
quarters of beginning, inexperienced teachers, “eluding thosi 
took our summer intensive preparation program, who were gi^en 
toughest class in the school for their first-year a ssig™ent~the 
kind of class that even an experienced teacher has difficulty in 
handling. How do you control such initial assignments in your 

system? 

Albert Schiff: I'd be glad to answer the question as best 
I can, but X did suggest to the chairman that there may 1 be some spe- 
cific problem areas that I think would mainly affect New York, Chi 
c / g o, Philadelphia, etc., that we may want to discuss in a smaller 

group session. 

Basically, the policy of the Superintendent and the Board 
of Education, was to try insofar as possible to alternately Pla° e “ 
a school where vacancies exist a beginning teacher and an experi- 
enced teacher. Now in reality, of course, you have a system of 
voluntary and involuntary transfers. We have a policy known as a 
career transfer. At the end of the three-year probationary period, 
every teacher is subject to transfer in the Detroit schools. We 
also give a teacher at the end of the third year a higher salary in- 
crement instead of the normal salary increment. 
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Theodore Lang: I can clarify the question. I think the 

problem is not so much the school of assignment as the class within 
the school to which the new teacher is assigned. I feel that there 
are many good classes in disadvantaged areas which a beginning teach- 
er could handle adequately and there are very bad classes in privi- 
leged neighborhoods, with which a beginning teacher would have dif- 
ficulty. It is the character of the initial assignment which I am 
interested in learning to control because principals will take the 
experienced teacher and gradually move him to the better classes, 
with the residual problem classes being left for the beginning 
teacher . 



Arthur Klein: I'd like to turn the meeting back to the 

chairman, but I've been urgently requested by these two gentlemen 
to try to arrange with Mr. Denn's approval a rescheduling of a dis- 
cussion period between 8:30 and 9:30 tonight in this room of repre- 
sentatives of the large city school systems to discuss greater co- 
ordination and more study in depth of the problems that confront the 
large city. 



Session No. 5>: Criteria: Problems in Validating Teacher 

Selection Policies and Procedures 

Chairman: Paul Denn, Board of Examiners 

David G. Ryans, University of Hawaii 
Donald Medley, Educational Testing Service 
Harold Mitzel, Pennsylvania State University 



Paul Denn: Most of us are in agreement that we must make 

some professional attempt to select, in an objective and valid way, 
those whom we want to teach our children. I think when we go off 
into a discussion of projected needs of personnel, inner— and outer- 
city rivalry, teacher shortages and so on, we tend to forget that 
the individual teacher, into whose classroom we’re going to send our 
own child, is the one to whom we are entrusting the fate of that 
child. The fact that there are enormous problems to be encountered 
when we embark on such selection does not detract from our obliga- 
tion to make the effort. The problems may be complex and difficult 
but I, for one, believe that they are not insuperable. Like a 
second-place transportation agency, we must try harder. One major 
key, I believe, lies in research, in depth, more intensive research 
and, perhaps, more scientific research, to establish valid criteria 
for selection and to develop the instruments that are needed. 

We are fortunate in having three research men who have 
been working very extensively in this field. I’d like to introduce 
them and have them present their views in turn now. First, I’d like 
to introduce someone who is known, I believe, as the Dean of Educa- 
tional Research. He was president of the American Educational Re- 
search Association. He’s the author of a standard work on charac- 
teristics of teachers and, at present, he’s the Director of the Edu- 
cational Research Center of the University of Hawaii, a most desir- 
able place to be. Dr. David Ryans. 

David Ryans: Coming before you following the most gra- 

cious and flatteringly generous introduction, and knowing full well 
I do not live up to ny billing, I feel very humble--and also very 
reluctant to open my mouth at all. I should have stayed bpick at my 
place in the audience and rested on the laurels so kindly bestowed. 

There are other reasons for my feeling somewhat reluctant 
to pursue the topic around which I built the position paper submitted 
to those responsible for our meeting. One reason is that I fear what 
I may have to say will sound rather academic to some of youj and most 
of us think presentations of that sort are likely to be much more ab- 
stract than practical. I hope ny remarks can be completely sound, 
but I also hope they can be accepted as practical in nature. To me, 
what I have intended to talk about does not seem academic 3 I sincerely 
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hope it will not appear so to you. For I do indeed agree with those 
of you who have pointed out that in the public schools we must oper- 
ate in a context of reality. But, as has been. amply demonstrated by 
some of the persons participating in this meeting, sound research can 
be accomplished in the public school real world; we can operate in 
the context of reality and still specify teaching criteria, develop 
procedures for predicting those criteria, and test those predictors 
against our agreed-upon criteria. 

Someone once said Josiah Royce followed a procedure in lec- 
turing in which he first told his listeners what he was going to say, 
then told them what he had to say, and finally told them what he had 
saiu. I will not be as brash as to try to emulate Josiah Royce, even 
to the extent of patterning my outline after his formula. But I 
would like to begin (particularly since you do not have copies of my 
prepared paper) by giving an overview of what I want to say and what 
I tr±ed to cover in my prepared paper. I want to say a few words 
about what we mean when we talk about criteria and the criterion 
problem— and why it is important for teacher selection; and I want 
to say something about teacher selection as I think it should be 
planned in light of agreed-upon criteria; and I had intended to talk 
biiefly about some of the problems we face in trying to designate 
criteria and obtain estimates of criteria. (Note that I am not talk- 
ing about teacher selection procedures, as such, at the moment, but 
about the criterion problem and estimates of criteria of teaching 
against which we may judge the extent to which teacher selection pro- 
cedures are working.) My major position will be one I have taken a 
number of times in discussions of this sort — that the criterion prob- 
lem is at the same time the most important and the most neglected of 
the problems of teacher selection. This is true, I think, whether 
we are concerned with content validity, or predictive validity, or 
whatever approach we take to the problem of validation. 

I am going to read my prepared paper in part, inserting 
comments occasionally, and skipping about quite freely. I hope the 
excerpts will not have too much discontinuity. 

A good deal of confusion appears to exist with regard to 
just what we mean when the term criterion, or criteria, is employed. 

A criterion is simply a standard or bench-mark usef to pro- 
vide a frame of reference for judging or evaluating something It 
may be thought of as a model against which comparisons may b'„ made. 
Usually criteria evolve from common agreement about acceptable stand- 
ards — regulatory boards for insurance, public utilities, banking, 
contracting, and such operate with a set of agreed-upon standards 
(criteria) as a model. In many circumstances (I think I should say 
in all circumstances) criteria are arbitrary and relative to values 
that are held to be important by some particular group of persons at 
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at some particular time and place. Indeed* the matter of ’•values’* 
and ’’value systems” is basic to the consideration of criteria. 

Now in relation to teacher selection, just as preferences 
and opinions (values) of individuals vary with regard to the compe- 
tencies and behaviors expected of teachers, so the criteria against 
which teacher selection procedures should be compared will often 
vary (at least in certain features) from community to community; and 
validity studies of teacher selection nearly always require replica- 
tion adapted to the varying conditions. 

In taking this position that criteria are determined by 
value contexts that differ among schools and communities, I am imply- 
ing that the first step in the consideration of criteria against 
which to judge a teacher selection program must be to determine the 
expectations that are held locally with regard to teaching and teach- 
er behavior. The extent to which there may be concensus about major 
issues, the greater the assurance with which a school administration 
or its educational researchers may approach the designation of cri- 
teria and their components, and the greater the possibility of con- 
ducting meaningful validity studies of teacher selection procedures. 

I shall return to the relationship between criteria and 
value systems later, but I first would like to comment further on the 
nature of criteria-- *and the frequent neglect of, and confusions about, 
considerations relating to criteria. 

Here I do not restrict my remarks to studies of teacher se- 
lection and research on teacher behavior; they are no more vulner- 
able than a great deal of research in the behavioral sciences where 
the problem of the dependent variable, or the criterion, has been ne- 
glected. From years of reading research reports and research propos- 
als, I conclude that many otherwise elegantly designed researches — . 
well designed from the standpoint of sampling, control of the experi- 
mental v ar iable and other independent variables, data analysis (and 
often involving real ingenuity of approach) — have, almost as an 
after-thought it seems, settled upon some available instrument, or 
perhaps hastily thrown together some test, inventory, . or other ob- 
servational technique without great regard to its validity and re- 
liability-proceeding on the assumption that such an instrument 
satisfactorily reflected, and provided useful estimates of, the cri- 
terion behavior. (A friend of mine sometimes refers to this sort, 
of thing as the "law of the instrument’’ and points out it is appli- 
cable in many of our activities. Ke illustrates his ’’law of the 
instrument” by saying if you give a baby a hammer, the baby will find 
something to hit with it; or, as it might be applied to the personnel 
field, if we have an available test or instrument of some sort, we 

find some place to use it— —often without regard to what the pur- 
pose of our work may be.) This happens to be a pet peeve of mine— 
the fact that so many investigators seem to neglect or give too little 
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attention to control of the dependent variable (criterion behavior) 
and that instead of considering this important problem from the very- 
beginning of their research, they appear to give it only cursory at- 
tention some time later in the investigation. 

I must move along. The second part of my paper had to do 
with a rather vague section titled, "Teacher Selection in Perspec- 
tive." In this section what I tried to do was to take the problem of 
the curriculum developer and think of the procedures we usually ap- 
prove in carefully carried out curriculum development. I then tried 
to look at teacher selection research within the same general con- 
text of developing procedures to conform to certain objectives,fol- 
lowed by evaluating the procedures against estimates of the attain 
ment of the selected objectives. Just as the curriculum constructor 
properly first sets up objectives, then spells out those objectives 
operationally, then develops the curriculum materials, and then eval- 
uates the curriculum materials against the operationally e ° “ 

jectives, it seems to me the logical process for teacher selection 
personnel is to set up the criteria of teaching, spell out those 
criteria objectively in terms of teacher behaviors, develop predictor 
procedures that are hypothesized to be related to the e “ 

haviors, and finally evaluate the predictor materials agai s 
criteria. In ny paper I try to spell out in some ten steps the 
operations one might appropriately employ. These invade. 

(1) Selection and de^ ’^nation of general aspects of the value 
svstem framework of the school/community as they relate to . 
teacher behavior. I am referring here to the agreed-upon quali- 
ties that are desired, or expected, of teachers in a particular 
place and in particular kinds of teaching situations. (Note, 
again that this process of arriving at criteria necessarily is 
subjective and a matter of the values individuals or groups o 
individuals may possess in common. When we designate criteria 
we proceed from a context of an accepted value. sys em. e. 
view teacher behavior in light of a set of attitudes, opinions, 
and viewpoints that reflect the sorts of teacher behavior we 
approve La prefer and also the kinds of behavior we disapprove 
and find unacceptable. To the extent any group of persons 
share in common certain expectancies, preferences, or biases 
about teachers and teaching, criteria of teacher behavior may be 
defined for that particular group.) 

(2) Identification of observable properties of. teacher class- 
room behavior that may be related to the specified operational y 
described criteria (i.e., the descriptive cataloging of teacher 
characteristics and behaviors that occur in the classroom). 

(3) Identification of kinds of situations in which the agreed- 
upon "valued teachers behaviors" may occur— and in which they 
may be observed and assessed. 
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(U) Operational description (i.e., description in terms of ac- 
tual teacher behaviors) of the agreed-upon valued behaviors 
that are to comprise the criteria of teacher behavior. 

(£) Selection of methods of estimating the operationally (i.e., 
behaviorally) described valued behaviors. This is the problem 
of instrumentation relative to the criterion behavior and ob- 
taining assessments of the criterion behaviors. In assessing 
some aspect or characteristic of the criterion behavior of 
teachers we are trying to estimate the extent to which that de- 
fined characteristic is manifest by some teacher. 

(6) Development of selection instruments and procedures that 
are hypothesized to yield estimates that will reflect the opera- 
tionally described teacher behaviors (criterion behaviors) — 
which, in turn, are assumed to reflect the value framework of 
the school and the community served. 

(7) Assembly of data yielded by the teacher selection instru- 
ments and procedures noted in Step 6 above. 

(8) Assembly of data yielded by the procedures used to estimate 
the criterion behaviors — Step li above. 

(9) Analysis of relationships between estimates of the be- 
haviorally defined criterion behavior and the estimates of 
teacher characteristics used in the teacher selection procedure. 

(10) Evaluation of the teacher selection procedures by the 
drawing of inferences about the validity of those procedures 
for predicting the criterion behaviors designated in Steps 1 
through 3 above. 

The next section of my paper deals with "Common Confusions 
in Dealing with Aspects of the Criterion Problem." I think I shall 
skip most of this, but I do want to mention the importance (even 
though it may seem academic) of distinguishing between criteria and 
"estimates of criteria." I used two specific examples in my paper 
to try to illustrate the distinction, one relating to "classroom 
management" and the other to "communication of knowledge." In the 
case of "communication of content" I suggested a few of many cri- 
terion behaviors that might be designated (e.g., clarity of presenta- 
tion, subject matter depth, etc.), all of these having to do with 
definition of sub-behaviors involved in the behavior we might call 
"classroom communication." In light of the criterion behaviors thus 
designated, I then suggested several different ways we might go 
about estimating such teacher behaviors — estimating the criteria to 
get the benchmarks against which we might judge our teacher selection 
procedures if we chose. Thus, we might employ observation^ procedures 
to rate samples of teacher communication behaviors or we might make 



18U 







frequency counts 5 or instead of resorting to a sample of the cri- 
terion behavior of the teacher per se, we might choose to view the 
criterion in terms of known or assumed correlates of "teacher com- 
munication of knowledge" revealed from measurements of the pupils. 

The next section of my paper dealt with "Some Considera- 
tions in the Designation and Estimation of Criteria," and in this 
discussion I was primarily concerned with the need for trying to 
avoid bias (i.e., criterion bias) that frequently affects both the 
definition of criteria and also the development of measures or 
measuring instruments that yield estimates of the criteria. I will 
not dwell on this, but I think we should remember at. least two very 
important sources of criterion bias, one having to do with the ex- 
clusiveness of our criteria (and the estimates we later develop to 
reflect those criteria) and the inclusiveness of our criteria and 
criterion estimates. We do not want to include anything in our 
definition or estimation procedure that is not really a part of 
teaching behavior (that may contaminate our thinking), nor do we 
want to exclude anything that is really a part of teaching behavior. 

We could give many examples of criterion bias with respect to 
teacher selection procedures. 

In subsequent sections of my paper I commented briefly 
upon approaches to criterion definition and also approaches to ob- 
taining criterion data. 

May I remind you again that I have been talking almost en- 
tirely about the designation of criteria and of the development of 
estimates of criterion behavior in teaching — not about teacher selec- 
tion procedures. I have pointed out that ideally this should pre- 
cede the development and/or choice of teacher selection procedures 
and devices and that such teacher selection procedures as might be 
adopted would be ones presumably geared to the criteria. And in the 
ten-step paradigm it was not until steps 7 > 8, 9 , and 10 that we got 
along with research directed at determining whether or not our selec- 
tion procedures appeared to be doing what they were intended to do. 

In practice this is not likely to be the way things work 
out. Very often we decide upon certain teacher selection instru- 
ments or procedures with only a vague idea of what criteria we are 
concerned with. To be sure the selection of devices has often been 
accomplished on the basis of considerable experience and wisdom grow- 
ing out of experience. But it is after the use of the instruments 
that we finally begin to wonder whether by chance they may be measur- 
ing the criteria we find acceptable in the community and school sys- 
tem. So, very often we enter my paradigm at about step 6 and then go 
back and retroactively try to think, "Well now, what was it we really 
wanted to be getting at, with these devices?" Perhaps this is justi- 
fied, or at least it is what we have to do in our "context of reality," 



but some Place along the line, even if it is done in retrospect we 
need to try to designate exactly what it is that we expect of teach 
ers and that we hope our predictors may predict. 

As I come to the conclusion of my remarks, I feel a strong 
sense of inadequacy. I know I have gone over this much too hurrie y, 
and I wish we could sit down for several days just to talk about how 
in a particular school system we might go about outlining a study 
which would start with the designation of criteria and involve the 
matching of predictor data provided by teacher selection procedures 
against operationally defined criterion behaviors of teachers ap- 
proved in the particular community. Perhaps some day we can do this. 

May I read the last hundred words or so of my paper. 

As is the case with most of you present, I have given a 
great deal of thought to the problem of the criterion, particularly 
as it relates to teacher behavior and to the problem of validity 
study of teacher selection devices. I find it easy to identify an 
recognize many of the problems and difficulties with which we are 
faced in trying to develop satisfactory descriptions of the cri- 
terion behavior of teachers and techniques which will yield va 1 
estimates of the criterion behavior involved in teaching. I recog- 
nize the sources of bias in the description of criterion ^avior 
and the conditions making for invalidity of the estimates yield e 
by different methods of assessing criterion behavior .But I am ad- 
mittedly frustrated by the difficulties involved in obtaining cri- 
terion data which are, on one hand, inclusive and compl ete and, on 
the other, exclusive and free of contamination . I know it is not 
easy to lick these problems, particularly when we must frequently 
conduct validity studies in situations where we have been using cer- 
tain teacher selection devices that were selected on an a prion 
basis without the benefit of guidance of adequate criterion descrip- 
tions, and now, after the fact, we are faced with the problem of . 
providing procedures that will yield estimates of criterion descrip- 
tions against which to test our selection data. I do not think 
situation is an impossible one, but I cannot help but recognise as 
I think most of us must, that we are faced with practical, considera 
tions which force us to compromise and employ m f ke ‘ s ^ 1 ^ methods 
that preclude the carrying out of validity studies of the quality we 

would like. 

Paul Denn: The next two panelists are not unfamiliar with 

the problems of education in large cities . They both were with the 
Board of Higher Education in New York City for a period of time. 
Donald Medley, who is the next panelist, is now head of the Teacher 
Behavior Research Group of the Educational Testing Service in Prince- 
ton. 
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Donald Medley (raising microphone a few inches): The dis- 

cussion is going to go on at a little higher level from now on— in. 
one sense, anyhow. I don't know whether I like to be here discussing 
problems which, as Paul has mentioned, Harold Mitzel and I have al- 
ready spent several years on while we were doing research in teaching 
in New York City. Maybe this is an admission that we left in defeat. 

(Dr. Medley now read his position paper which is reproduced 
on page 9 5 • ) 

As I've been sitting here, I've been thinking that this 
proposal is couched in terms of the present structure of a teacher's 
career where a teacher goes to school so many years and then the 
training stops and she teaches so many years. Apparently, the theory 
is that at some point a teacher becomes competent. If you ca *\^ us 
get him to that point, then you have it made. I think if we thought 
of teaching and teaching competences as something that grows through- 
out the entire career, and if we thought of the selection problem as 
identifying steps, when a teacher has learned all that a college can 
teach him, let's take him into the school system, and when he has 
learned all that we can teach him in the school system, let's make 
up our mind whether we want to keep him or not. Now if I had my way, 
this distinction between what happens in a college and what happens 
in the school system wouldn't be nearly as clear-cut. Something like 
what Harry Rivlin suggested in his paper at the Great Cities Con- 
ference and what I think he ' s trying to implement at . Fordham where 
the transition from an undergraduate pre-service training to in- 
service training performance is more or less much more gradual and it 
is much more individually tailored to the status of the individua . 

So in thinking about this proposal, don't throw it out because you 
think it won't work under the present system. After all, we're en- 
titled to talk about what ought to be as well as what we think migh 

be. 

Paul Denn: The next panelist, Harold Mitzel, is Assistant 

Dean of Research at Pennsylvania State University. 

Harold Mitzel: I'm reminded of the military convoy situa- 

tion. You know, a military convoy travels at a constant rate of 35 
miles per hour, at least that's what the convoy commander has set 
when he takes off in the lead jeep. But it always happens that the 
guv who's on the rear end of the convoy drives at a constant speed 01 
83 miles per hour just in order to keep up. So if I seem to be going 
83 miles per hour, it's because I'm trying to catch up with this 
convov. It's a pleasure to share another forum with Dave Ryans and 
Don Medley. I've had the joy of working with them in making pres en a- 
tions in the literature, and an appearance before a group like this 
is another interesting adventure for me. I think it is significant 
that we all prepared our papers independently, but some of the same 
threads or themes do run through them. 
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(Dr. Mitzel now read his position paper which is reproduced 
on page 99.) 



Discussion 

Paul Denns Would you like to ask some questions, or make 
some comment? Who would like to begin? 

Douglas Bray: I’d like to make a contentious comment. I 

feel a great contrast between the severe and immediate practical prob- 
lems in places like Detroit, Los Angeles, and New York and the formal- 
istic academic research suggested by this panel. These two worlds 
are just so far apart that very little progress will be made in solv- 
ing problems of teacher selection if we insist on the kind of purity 
which has held back progress in this field for so many years. I 
really feel that something far more quickly workable is called for. 

David Ryans: I don’t think it's nearly as impractical as 

it sounds. No. I can really get kind of worked up about this be- 
cause you know we talk about all of these practical problems that 
face us, and we forget about the fact that we can sample, as Harold 
was saying and as Don was saying here. We can get samples of teach- 
ers and we can work with them, and this doesn't mean that we have to 
do this with everybody. We can carry on a parallel process of, call 
it expediency if you will, so far as teacher selection is concerned, 
at least this is my contention and one of research which does have 
some foundation. You see, I could be contentious and say as long as 
you keep on doing thnngs the way that you're doing now, you don't 
know what the dickens is happening and never will because you don't 
have any criteria to stai*t with; you never have anything to test 
them against. 

Douglas Bray: I heard someone, I think it was Harold Hunt, 

say that there are a lot of "izes" that the administrator has to use: 
supervise, organize, and finally, compromise - with the last of these 
perhaps being the most important. I think we can do some compromis- 
ing and at the same time achieve a good part of our goal here. 

* 9 

Samuel Brownell: It seems to me that in our discussion of 

teacher selection we're not so far apart as what might be suggested. 
Obviously, we need instruments. We need various ways to know how to 
put the teacher in the best place. We don't always have the chance 
to do that in a school system, so we do need help in that. 

If we are in a small school system, and I used to be a 
superintendent of a small school system (that was before I went to 
Detroit) and I knew that I had to fill a vacancy in a third grade in 
a particular school, the criteria of the kind of teacher needed could 
be pretty clear because I knew the administrator. I knew I had a 
stable situation. 



When we get to the matter of selecting people for a large 
city school system, I think we have to recognize the fact that what 
we’re trying to do is to find people who can come into a school sys- 
tem, not a particular school. We’re looking for people who are go- 
ing to make a career, who can be placed in one school or another. 

In other words, we're looking for people who are flexible, who have 
the general basic qualifications, and then we can fit into many 
types of situations. Furthermore, we have to recognize the fact 
that people change. A teacher that we select today isn't the same 
as the teacher next year, nor ten years from now, nor twenty years 
from now. What we're trying to do in selecting teachers for a large 
school system is to find the people we ought to take into the large 
organization to work. 

It seems to me that the emphasis on the selection process 
has to be to protect the children against those who are obviously 
incompetent, who haven't the characteristics that we can take the 
chance on. We're trying to do the same thing the life insurance doc- 
tor does-- to get out the poorest risks. We may be wrong occasion- 
ally, just as they occasionally select poor risks. I'm just simply 
saying that I think if we take that into account, the fact that 
teachers are flexible and they do learn and they grow and they 
change— then we're not quite as far apart on this as it was sug- 
gested because we have two different kinds of things we're trying 
to do, and we need two different kinds of processing. 

Paul Denn: Sometimes some of us want more than that when 

our own children are involved. Dr. Baldwin? 

Wayland Baldwin: At one time I felt impatient when I 

listened to the "researchers." I wondered if it would ever be pos- 
sible for researchers and practitioners to speak the same language. 
Some two years ago I attended a £-day conference held in Tempe, 
Arizona. The topic for discussion was "Bridging the Gap Between Re- 
search and Practice." Before the five days had ended, researchers 
and practitioners were speaking the same language. I gained much 
from this conference. Particularly I was convinced that the re- 
search expert has much to offer to the personnel practitioner. It 
is important that we get together. It is not possible to bridge 
this gap in a 2-day conference. We need much more time. I should 
like to recommend a 6-week seminar for selected leaders in the field 
of educational research and school personnel administration. 

Paul Denn: Well, we can make a start, and let's try to do 

that because it's most important for us. Dr. Darland, you had your 
hand up. 



David Darland: I'm merely going to say that I think you 

are closer to the truch of where you ought to be in terms of what 
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this meeting is about than any time since I've arrived, and I heard 
Dr. Bray very clearly support the idea of criteria. Well, where did 
they get the one, the criteria that you use—did you just reach out 
and pull it out of the air? 

Jay Greene: I've a question, and I agree with Dave be- 

cause I sense here some closeness, a question to the three people 
who just spoke. Aren't there different criteria and different in- 
struments at different levels that should be perfected to achieve 
special purposes? For example, the first selection comes when a 
student is selected in the college to go into teacher training or 
teacher education. Now at that level there are criteria that may be 
different than the criteria that you want to describe later on. 

There should be criteria for putting them into this program and cer- 
tain instruments that ought to be set up at that time for that pur- 
pose. If we did that, we would be measuring achievement, as Dr. Med 
ley points out. Then the next step comes after he has graduated 
from this program and he wants to be a beginning teacher. Here you 
want some criteria related to the beginning teacher phase, and you 
want to try to perfect certain instruments limited only to the be- 
ginning teacher phase, because you can't measure the other aspects. 
Here again, you'd be measuring his achievement at that point which 
Dr. Medley says is more possible than getting into prediction. 



In the third level, after he has taught for a year and had 
some teaching experience, you're down to the real job itself. Then 
you want to determine some different criteria and at that time per- 
fect some instruments, perhaps the teaching test you talked about, 
which could be introduced perhaps for the first time. You canjt use 
it earlier and here, too, you are measuring on that level--achieve- 
ment rather than prediction — which may come in the total situation. 
My question is: "Is it possible that we can use some of the instru- 

ments that you would reject because they're not suitable for predic- 
tion but they are more suitable along the way and to develop dif- 
ferent sets of criteria?" 

Paul Denn: Harold, would you want to take that first be- 

cause you had presented some steps in the process. 

Donald Medley: I have a short answer, yes.. It seems to 

me what Drs. Bray and Brownell say sort of hooked up in my mind. 

When Dr. Brownell talked about what we're really trying to do, we're 
trying to identify teachers who can go into a large city system and 
be assigned any place in the system and succeed there, be flexible. 
Flexibility has been added to all the things that Harold talks about 
and suggests. I think the gap here is really between the job we're 
trying to do and the tools we f re trying to do it with. We*re trying 
to pick this kind of super-teacher with some kind of a paper-and- 
pencil test. 



O mmm 
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I remember someone used to say to me, if you want to tell 
what a cowboy is going to do when he' 3 drunk, find out what he did 
the last time he was drunk. And I think that we've just got to... if 
you want to attack this problem now, and you don't want to run away 
from it, you've got to get that teacher as close to that situation 
he's going to be in as you can and see how he does in it. 

Now we researchers are supposed to be attacking the basic 
problem and finding out enough about the teaching process so we can 
help you, but it's going to be a while. It's going to be a long 
while and in the meantime, that's about the best advice that I think 
we can give you from this side. But I do think that's what Dr. 

Greene has said several times, and every time I want to get up and 
agree with him. 

In the meantime, we can weed out some people, the people 
who never learned how to speak clearly enough to be heard in the 
class and the people who are obviously in such poor control of their 
own behavior that you could see that they couldn't achieve in a class- 
room. We could still do something like that, but look at what these 
people have now without bothering to prove that this is correlated 
with teacher effectiveness or not. One thing I know from all the re- 
search I've done and read, and one conclusion that's most sturdy is 
that there's no way to _ tell how a person .is going to perform in the 
classroom, outside of the classroom^ there's no test yet that will do 
it. I've got high hopes for simulations, but that's in the future, 
as you know. 

David Ryans: I just wanted to respond to say I agree also 

with what Dr. Greene said and make two comments. Talking about pre- 
diction research in any field, we often talk about, or we've just 
roughly classified and people have talked about this for years, im- 
mediate criteria and intermediate criteria and ultimate criteria 
that are further along. For example, what the medical doctor is ac- 
tually doing after he's out in practice 20 years as compared to pre- 
dicting his grades in medical school which would be immediate. 

I also wanted to add that in trying to predict, not cri- 
teria of teacher effectiveness but certain criteria that I chose to 
identify and try to predict, the further I went back, that is, did 
the predictive testing back in college days, the. lower the correla- 
tions got with the criteria measures that we obtained when they were 
actually teaching. On the other hand, we could get some pretty high 
cross validity coefficients by taking two different samples and could 
get some very high cross validity coefficients with teachers who were 
on the job and at the present stage of their performance. I'm just 
agreeing, that's all. 

Perry Kalick: I think that we can all agree that the pay- 

off is in the actual classroom performance of the teacher. The 
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Board of Examiners is aware of this too in their giving of the per- 
formance and classroom teaching tests. However, the fact that these 
tests are expensive and relatively quite time-consuming propositions 
probably accounts for their increasing disappearance from the teacher 
selection process. In addition, a candidate’s performance in a 
class of middle class children, might hardly be indicative of his per- 
formance in the difficult class to which he will be appointed. I 
think the practicality of the use of the teaching test rests upon 
the question of how many samples of teaching performance are neces- 
sary and the question of whether there are enough funds available 
for the tests. Furthermore, there is the problem of logistics; 
i.e., where are you going to give the examination, and how many 
qualified people do you have readily available to make the assess- 
ments? 



Dr. Brownell talks about not taking a chance. Can we af- 
ford to take a chance, particularly in view of the damage that is 
done in the first year of teaching? In the teacher selection proc- 
ess we have not yet been able to be predictive of teacher hostility 
in the classroom. Jersild would say that almost a classical analy- 
sis of a teacher is necessary to find out whether or not a teacher 
is going to do real damage, verbally and/or otherwise. In view of 
the cost of the latter, perhaps we should make pre-service teaching 
observation and the probationary period part of the selection process. 

Douglas Bray: These comments sort of jell one of my ob- 

servations to what I thought they were saying.. I think the real gain 
is to be made by knocking off the bottom end of the scale. If you 
can knock out the one- third of the candidates who are manifestly lack- 
ing, you would make a substantial practical gain. You don’t need 
these elaborate research designs to achieve this. One other comment 
on the criteria: many people seem to think that observation in the 

classroom would provide the best criteria just as in business people 
believe in training the supervisor to rate job performance. I’m not 
so sure, however, that simulation does not yield better criteria. 

You can not only have a highly trained observer group, but you can 
standardize the performance situation. 

David Darland : There may be another factor we should be 

considering. In business and industry when they find that the struc- 
ture is in the way of function, they change the structure. And they 
are much freer to do so. However, education is going to be re- 
organized. But I sense that statements being made here on the evalua- 
tion of teachers are based on the theory that the self-contained 
classroom is going to continue. Moreover, maybe the major cities of 
the United States are just simply unmanageable not only from the point 
of view of education but with respect to many problems. 

Paul Denn: Thank you, ladies and gentlemen. The meeting 

is adjourned until 8:30 p.m. 
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Session No. 6: Open Discussion 

Chairman: Paul Denn, Board of Examiners 



Paul Denn: Mr. Charles Cogen, President of the American 

Federation of Teachers, has had a tremendous amount of experience 
with respect to large urban education, certainly because the largest 
part of his following is located in these areas, and he is acquainted 
then at firsthand with a number of these problems. I ve asked him to 
get the ball rolling for us by telling us a few things about his ex- 
periences in large urban education and some of the problems as he 
sees them. Charles Cogen. 

Charles Cogen: Thank you, Paul. As a host and also as a 

former colleague, Mr. Denn asked, as we were entering the room here, 
whether I would be willing and ready to make a few comments, I sup- 
pose representing the teacher's point of view— how the teacher sees 
this problem of urban schools and along with it, of course, the 
staffing of the urban schools. 

Now, I've had my experiences in New York City, as you all 
know, as a teacher for many years and also as a lower . echelon super- 
visor, as a department chairman. But I've also come into contact 
with many teachers and others, such as professors who have been in 
touch with the problem in their own areas throughout the country. 

I think it is not so much a matter only of size of the 
school system that makes New York City so different from other 
cities, ‘in New York by the very magnitude of its problem, the geo- 
graphic situation, the ethnic problems, and so on, I think the prob- 
lem becomes a qualitative one as well as a quantitative one. 

Now from the point of view of the teacher . We are talking 
now about those who are thrown into a classroom situation right at 
the start of their careers; the teacher comes in, .in most cases, 
without any real practical guidance, and is, I think it is fair to 
say, very much in a fog. One of the comments I made earlier in. the 
evening applies very much here. There is not enough guidance given 
to the beginning teacher to set him at ease, to tide him over this 
critical stage. That confronts every teacher but in particular the 
teacher in the inner-city school. 

And you have the problems related to the disadvantaged 
children. The teacher comes across these problems, not only of cul- 
tural deprivations, but also of the under-privileged from. the point 
of view of emotional handicaps; also the problems of discipline that 
the teacher has to confront. This is something that every teacher 
has to face and is very much concerned about, as you know. And so I 
feel it is imperative that there be some way of overcoming the diffi- 
culties that the teacher has to confront when he comes into a setting 

of this kind. 
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Now I mentioned earlier in the evening the More Effective 
Schools program. I feel that something along those lines is impera- 
tive from the point of view of recruiting and retaining the teacher. 
We must have a setting where the teacher is able to give individual 
attention. I walked through a school the other day, one of the "more 
effective schools," and you would hardly believe that there were any 
real problems existent there among the children. The teacher has few 
children to handle, she has all kinds of assistance— teachers aides; 
teachers in training who really give some help, individual help; sup- 
plies of kinds that they never have elsewhere. These are things that 
help the teacher to overcome the great initial handicap which they 
suffer when they get into a disadvantaged school. And o£ course 
there is the problem of class size. 

I would say that just about as important as anything else 
is this matter of large class size. This is especially significant 
in the school with disadvantaged children. In such a school, where 
you have also large class sizes, the problem is aggravated to a much 
greater degree. And so long as we do not adequately face this prob- 
lem of class size, and also the problem which I haven* t mentioned 
yet, of salaries, you* re not going to get anywhere from the point of 
view of recruitment and holding on to the teacher. I was at the con- 
ference on the education of the disadvantaged a few months ago at 
the White House. There were all kinds of proposals that were made 
at the panels— remedial reading and ungraded classes, and so on and 
so forth, but you hardly heard anything at all, except from myself 
and iry colleague from the American Federation of Teachers, on the 
two big questions of salary and class size. .School systems are try- 
ing to run away from them, but I don*t see how you can escape them. 

And finally— not finally by way of covering the subject by 
any means, but from the point of view of the time limitation that 
has been imposed and that I want to impose upon myself — I think the 
problem of adequate supervision is extremely important. We need 
supervision that is democratic, that is helpful, and constructive; 
and I do not use these terms in the loose sort of way that is often 
used in college training classes. You really have to have a super- 
visor who is on the job with the teacher. I think one of the things 
that the Board of Education might consider in any city would be for 
a supervisor to do some teaching as well as supervising, just the 
same as a department chairman does in the New York City schools. 

The supervisor needs to keep his finger in the pie, so to speak, to 
realize what the problems are, to be really working on a colleague 
basis with his teachers rather than on a basis of superior towards 
inferior . 

Now I know there are many other problems, but I think 
these are the major ones that teachers throughout the country in the 
large cities are thinking about and worrying about 1 . Of course, you 
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have the matters of non-teaching chores and the duty free lunch 
periods which are gradually being taken care of by col .ective bar- 
gaining procedures. I thirk this is the wave of the future, ohe 
collective bargaining that has begun in the major cities and has 
gone into the smaller communities as well, where teachers and ad- 
ministrators together work through their problems and get to a com- 
mon understanding, rather than having decisions imposed upon the 
teachers from above without any clear thinking through b ° h 

sides of the table. These are some of the things that I think we 
ought to keep in mind when we consider the problems of recruiting 

and retaining teachers. 

Fred Williams: Mr. Cogen gave the point of view of the 

problem that the teacher faces in the school of the disadvantaged and 
I would just like to be presumptuous enough to give the point of view 
of the problem the disadvantaged child faces with respect to some 
the teachers we have who are not able to develop the kind of rappor 
necessary to do an effective job teaching that child. 

Thus, when you mention ineducability with respect to the 
child as one of the problems, I would say that the problem takes 
place in reverse also. Sometimes we are not successful in giving 
the teacher whatever is necessary for that teacher to function suc- 
cessfully. This leads me to the heart of what I want, to mention— 
that is, those elements that were alluded to earlier in the day, 
and which are considered to be essential for success in the inner- 
citv. A couple of speakers suggested that we should look to the 
placement of the teacher in a specific situation because some seemed 
better able to function in the inner-city. This capacity is not one 
of the elements that is measured and I «m wondering if this is some- 
thing to which we should turn our attention. What are these particu- 
lar characteristics? Are they worth measuring or are we measuring 
them or can we measure them? Are they more significant than some of 
the things we are measuring? Are there any of them that are racially 

associated? 

I don’t want to suggest that there is any necessary corre- 
lation between color and these particular characteristics but, as a 
general premise, has the nature of the society in which we have 
lived created a condition which would allow a minority group . teacher 
to have these characteristics for functioning in the * 

greater extent than would be true of someone who is not of the minor- 
ity group? This leads then to the next items, namely, the degree to 
which large cities have been and are successful in obtaining minority 
group members on staff, and how much effort is being made in staff 
integration. 

Paul Denn: Well, Fred, you raised two questions in ny 

mind. One is, can we train a middle class teacher, a teacher with a 
middle class background, regardless of any racial implications 
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because there are middle class people of all nationalities and races 
to teach the children iri the inner-city. And the other question is 
raised from an implication I got from what you say, can a teacher 
brought up in a minority environment teach in a middle class area? 

What I’m suggesting is that I think that both are possible. Would 
you agree or would you disagree? 

Fred Williams: Paul, yes, I think that both are possible. 

What I am saying is I wonder if someone has a head start because he 
has lived a life more closely akin to these youngsters, either throug 
economic circumstances or because of racial identification. Is such a 
person more likely to have the particular characteristics which would 
make him more successful at least in the development of rapport. I m 
not talking now about methodology or content knowledge and so on. 

This is the point that I'm raising. 

Paul Denn: I give the first speaker the prerogative. 

Charles Cogen. Just one second. I think at one point I 
referred to the group as being -neducable; I really didn’t mean it 
that way. Ineducable from where they stand under the given condi- 
tions but certainly not from a sound psychological and wholesome 
point of view. There is then the underprivileged and this is what 
I meant, they haven’t gotten their proper orientation to the setting 
that they are put in and we have to give them this compensatory 
education, head start, and so on. 



Harry Gilbert: It was because we realized there was going 

to be a good deal of attention to the problems of the culturally dif- 
ferent that we invited to this conference a social psychologist and a 
sociologist. Each one turned us down after long deliberation, so we 
were stuck at the end and didn't have the representation of sociolo- 
gists or social psychologists here. I want to make a few comments in 
this area, however. I feel very strongly that we're doing a very- 
great disservice in our labels of the culturally different as cultur- 
ally deprived, etc., and we're being downright racist, actually, in 
assuming a kind of homogeneity among these groups. 



A very simple study that I've alluded to many, many times, 
conducted by our New York City Board of Education, seems to me a 
very great big eye-opener. I'm talking about Joe Justman's simple 
study of pupils he called stable and mobile. I'd like to take a 
minute or two to tell you about this because I think it nails down 
this great point of thinking we're dealing with a homogeneous group. 



Justman took the youngsters in completely Negro schools in 
Harlem and Bedford Stuyvesant who were in the sixth grade . _ He found 
that about half of them had been in the very same school since the 
third grade, so he had records on these youngsters from third to 
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sixth grade in the same school. He found that the mean IQ for this 
group, which he called the stable group because they were in the same 
school, was about 100.8 in the third grade and about 102 . k in the 
sixth grade, something we like to call a diiference that is signifi- 
cant statistically but operationally not much .different. But for the 
other youngsters, the other half, who were in the sixth grade but had 
transferred one, two, three or more times from third to sixth grade, 
he found that the initial IQ was 98 something and then 9h in the. 
sixth grade. When he analyzed that further, he found that the kids 
who were transferred four or more times had gone from something like 
88 to 8U . The point is that the more transfers, the lower the IQ 
initially and the lower it becomes in the sixth grade. He repeated 
this study with achievement tests and got the very same results. 

Now this is a simple statement of finding. These are facts; 
he found that these kids whom he called stable had an IQ and achieve- 
ment just like the rest of the city, but the kids who were mobile 
started lower and kept getting progressively lower. Now when. you take 
this whole group and put it together, you get the common finding 
that has been repeated over and over again, in other words... 

Paul Denn; Harry, if I may interrupt. . .If you had taken 
the statistics only for the sixth grade, without analyzing them out 
or going back, it would look as if the whole group had regressed. 

Harry Gilbert: And this is the common point that is re- 

peated over and over again, that the youngsters in the schools in the 
slums start out lower and keep dropping lower. .1 think there's a 
massing effect that occurs there, and I submit it's a very racist 
kind of massing and something that should not be done. Now I'm cer- 
tainly for our trying to see what teachers are successful for what 
children, but I think we had better be very careful of our analyzing 
the youngsters and not treating them as though they are a homogeneous 
group. This is a blind alley. 

When Charley Cogen was talking about what we need, he men- 
tioned all these things in the more effective schools, and a plague 
on that name, this is giving it a label and it's saying it's success- 
ful before it even is evaluated. I'm sick of calling projects 
"Higher Horizons" and "More Effective Schools" before we've evaluated 
them. By golly, you and I know that some of these schools are more 
effective than others, and I know that for some of the kids, the 
schools have been effective, for some they are not, and I'm a little 
tired of the way we've compressed and homogenized all the. results to- 
gether. What I'm suggesting is that the dimensions of this problem 
are sc very deep and so very complicated. 

I know something about these mobile kids because I was a 
school psychologist, and I've been in their schools and their homes. 

I think the schools don't know how to treat these children. I don't 



care what kind of teacher you* re going to get, you cannot substitute 
for the dire effects of the deep social pathology . This has a long 
history, economics, political, social and there are a lot of reasons 
that are societal as well as personal and familial. 

But if we're going to expect the schools to be able to 
treat these youngsters, then I think we're starting out with a bar- 
ggjji ye ' 11 never be able to fulfill « X think we better get read^ to 
face the fact that we need some real social dynamics that are of an 
order that is unacceptable yet. That means taking these kids away 
from the home, and I've tried this. I have been in court on a ne- 
glect charge in a home where there was drunkenness, and the kids were 
absolutely terribly neglected. A smart lawyer made a monkey of me 
when he had the mother enter court in a nice black dress with a white 
handkerchief and a lot of tears. You can't take kids away from 
mothers, and any smart lawyer can make any foolish psychologist look 

even more foolish. 

This is why I say we've got to do some things that are 
daring and different. I am greatly concerned about all of our pro- 
grams, our head starts or whatever we do where we're trying things 
for large groups without really being analytical and diagnostic 
about what we're doing for whom. Excuse me; this is a topic on 
which I feel very greatly, and I can't help expressing it. 

Paul Denn: You started by saying that we seem to have com- 

mon problems all over the country wherever we have a large urban cen- 
ter and let's see whether or not that's so. Let me ask, for example, 
some of the people from around the country, what do you concede to 
be your most serious problem as far as urban education is concerned 
in your area? Let's start with Detroit. Alj would you? 

Albert Schiff : I have listened with great interest to 

Fred Williams and also Charles Cogen's statement of problems as they 
view them. I'm not going to go into Harry Gilbert's now because 
Harry is dealing with a very small select sample of the thousands of 
youngsters we have in our classroom. In Detroit, the immediate prob- 
lem we have is staffing classrooms with able, competent teachers. 

This is the problem. We have only two sources of teachers, (1) the 
beginning teacher, (2) the experienced teacher. If we have vacancies 
in the inner-city that we have not been able to staff, the Negro com- 
munity would say, "See, how can we have quality education when you 
don't even give us a classroom teacher?^ 

It becomes a racial issue as well as a quality education 
issue. Out in the peripheral city area, if we can't find a math 
teacher, the community may accept it as a shortage of math teachers, 
but that same shortage in the inner-city has other overtones. There 
is no question that the Negro community can show that we have more 
substitutes filling regular positions in the inner-city than we have 
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in the peripheral areas. And it doesn't make any difference what 
logical explanations you can give, such as "teachers still have the 
right to select the places where they want to teach." In the final 
analysis you can say, "Here is your assignment," but they still have 
a right to accept or reject that assignment. In Detroit, we place 
teachers under certain policies and procedures and Ted Lang raised a 
question about principals. Principals have little to say in the 
placement of teachers in Detroit. All a principal can indicate o a 
placement officer is his need for a math teacher. He cannot tell us, 
"I know somebody who is asking for a transfer to my school and I 
want you to give him that assignment." 

Detroit operates its assignment policy — both initial ap- 
pointments and transfers — in accordance with the "balanced staff 
concept." This means- every attempt is made to have a well-balanced 
staff as it pertains to beginning and experienced teachers, race and 
sex. Detroit has an annual racial census. The last census in Octo- 
ber showed that all schools in Detroit are racially integrated. We 
currently have 31 psr cent total instructional staff who are Negro. 
This is why I look with great trepidation upon the theory that Ne- 
groes can best perform only in the area where they have a better 
knowledge of the culture of the Negro. Now, I don't know whether 
this is what Fred had in mind— but I am sure you can't have real in- 
tegration of staff by limiting the placement of Negroes only in 
Negro schools. 



The majority of Negro teachers are still in predominantly 
Negro areas* Our problem is to get more experienced teachers to move 
into the irtner-city— and I hope the local chapters of the American 
Federation of Teachers and the National Education Association will 
help us in this area. What I am really saying is the time has come 
now when teacher organizations, who have a stake in education, must 
convince their members to help in the inner-city where experienced 
teachers are needed. Perhaps, a trial basis for a period of ^wo 
years with the understanding that if it doesn't work out, the teacher 
can be returned to his former school. New York tried voluntary 
transfers and only a half dozen people or so were willing to do this. 



In Detroit a condition of promotion to an administrative 
position is experience in the inner-city. This works all right for 
the few seeking promotion, but it still doesn't take care of the 
larger critical need to really staff the schools with experienced 
teacVers in greater numbers. 



Now, a word about actual placement. Here is a school that 
has two Negro teachers out of a staff of forty. That is not a truly 
integrated school. Not when 30 per cent of your teachers are Negro. 
There is now a vacancy in that school. We have eligibility lists 
a social studies vacancy exists in that school. We have two sources: 
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(1) a transfer list of people who have asked for transfers, or (2) 
a beginning teacher list. We make the judgment to place a Negro in 
that school. In order to give this school racial balance we go to 
our transfer list until we reach the first Negro who has asked for a 
transfer to that area and we offer it to him. Or we go down our eli- 
gibility list for beginning teachers and we pick the first Negro to 
fill that spot. No if's, and's, or but's about it. This is the 
policy of the Board of Education to achieve a balanced staff. Racial 
balance is one of the factors in placement and you're not going to 

get racial integration if you side step these issues. 

% 4 

I think it's important that the teacher organizations help 
us. They want to help, but not when it's at the cost of teacher 
morale. When they start talking about what's good for the system 
and about the need for smaller class size and other supportive serv- 
ices, our reply should be these kids are without teachers now and 
they are in your school system. Can't we get the experienced teach- 
ers to go to these schools and help us during this emergency, be- 
cause we can get all the beginning teachers we want to go into the 
more privileged areas. This is an easy thing to do. 

I think I've spoken enough. I feel so strongly about this 
because people talk about what they want to do but when the chips 
are down, nobody is willing to take the stand and the personnel 
administrator is in the middle of it; the principal, God bless him, 
is in the middle of it; and the teacher is certainly in the middle of 

it. 

Harry Gilbert: How do you know the race of a teacher? 

Albert Schiff: Although we do not code teachers by race, 

almost every teacher whom you employ comes in for interview or as- 
signment. Now we know that if you have transcripts from some south- 
ern universities or the teacher had attended a segregated school, we 
can assume we are not mistaken about race. Sure, we miss sometimes 
and often we make mistakes— we've sent some white teachers when we 
really thought they were Negro teachers— but we were right in about 
90 per cent of the cases. By the way, Harry, there is nothing in the 
law that prevents the coding of teachers after employment. In other 
words, race cannot be a condition of employment, but we can code our 
teachers any time after employment. We don't do it in Detroit. 

Many of these people will be seeking promotion later on so even if 
the law provides that we may code them, we feel it prudent not to do 

so. 

Paul Denn: Okay. Let's give Fred a chance to get in on 

this. 

Fred Williams: I just want to respond to a point that you 

raised, Al* 
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I'm not in favor of the all black schools, staff and pupils 
or anything else, nor am I suggesting that only the Negro can teach a 
Negro child by any means. What I am saying is that because of the 
kind of society we have had, unfortunately, it may be that a Negro 
teacher, generally, may have a greater likelihood of developing rap- 
port, and if this is an important characteristic, then we ought to 
try to tap it to the extent possible in getting this characteristic 
into our schools and taking advantage of it. 

Paul Denn: Fred, I go for what Al said, namely, if we 

spread them around the schools. I think it's just as important for 
the white youngster to know the Negro teachers. 

Fred Williams: Believe me, I've devoted much effort to 

this essential aspect and it is why I raised the corollary question 
of staff integration. 

Paul Denn: As one of my friends used to say, "Let's not 

waste time in heated agreement." Jules? 

Jules Kolodny: There are sc many things kicking around here, 

I don't know where to begin; but, I'm going to start with the fellow 
at my right. 

I would like to remind you, Al, that when you say you have 
an emergency today and cannot organize a program of smaller classes 
now because it will require five or ten years to develop, I would re- 
mind you that the Federations with which I am familiar tried to get 
you ready for today five years ago and ten years ago; and you didn't 
listen. When I say "you," I don't mean Detroit; I mean New York, 

Los Angeles, etc. People suddenly get excited about Watts when the 
thing explodes. But the Los Angeles Federation of Teachers has been 
talking about inadequate education there for more than a decade, so 
that it isn't quite fair to say that teachers aren't prepared to go 
with you, when you have allowed the situation to deteriorate. 

I would like to go to the question of race and faculty in-, 
tegration. I sat down with a committee headed by Fred Williams and 
a few others. We worked out a plan by which our Central Placement 
Bureau would try to identify Negroes by virtue of the fact that we 
know we have ghettoes and even if you have a Bachelor's Degree, you 
are more likely to live in the ghetto or on the periphery of the 
ghetto as it moves north or south into a white community. And. I 
think Ted would agree with me that the procedure worked fairly well. 

I don't think we had 100 per cent success; but we were able to iden- 
tify Negroes by virtue of their address. This would include the 
graduates of southern colleges and the city college graduate, as 
well as the New York University graduate who lived at a certain ad- 
dress which was in a deeply populated Negro community. 
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Now, earlier in the day I talked about changing the courses 
that are required for teachers, and somebody took me to task ““ 
said I wasn't abreast of the times. Let me give you an idea of wha 
I mean. I think this comes back to the type of thing that Ted has 
talking about. I'm going to use a very simple i Lliust ration with- 
in the Puerto Rican community. A Puerto Rican child is ta gh 
infancy on that when his elders scold him or take him to task * 01 
misbehavior, he should hide his head in shame. Then he comes to New 
York lives in a Puerto Rican section of our city and the teacher 
scolds him He holds his head low in shame, and what does the average 
whit^middie class teacher say to such a child? "When I'm talking to 
vou I want you to look me straight in the face." This is a cultura 
dynLnic which is completely misunderstood with the best of intent ions 
bv the teacher. I use this as a very simple illustration of the kind 
of program °Iwould like to see embodied in teacher training institutes 
^n terms of group dynamics, cultural sociology, psychology , minority 
culture! history P a i^ n understanding of things of ^e sort l described. 

I still think there's room for changes in preparation, despite the 
fact I was told that I'm not abreast of the tiroes. 

Let's go on a little further. We talked of smaller classes 
and life isn't quite that simple. Perhaps, if ten years ago when we 

talked of Smaller classes, we'd be ready in 1966. But "^/have 
of smaller classes in 1966, the answer given is that we don t ha 
the space and we don't have the capital budget to do this. Now if 
vou had the space, you would, by making your classes smaller, have a 
greater personnel problem because you'd need considerably more teach- 
!rs. We recognize this problem. Perhaps what we ought to do is go 
into another kind of thinking. Try to keep a relationship of maybe 
three teachers and two classes, so that one is free to re ax, , 
free to pull three or four or six youngsters out lor indivi a _ “ 
tention during parts of the day, etc. Because we are dea ing 
totality as you affect one aspect of the problem, the reduction of 
class size! you impinge upon the lack of supply for personnel. But, 
that doesn't mean we have to throw in the sponge. We ought o give 
thought to new devices and I think that there are new devices. 

Let me go on to the kind of teacher who is sent into schools 

in the disadvantaged areas. Teachers are 1 “'S 

and they suffer from all the same disabilities and hostilities and 
hatreds! There is a small percentage of teachers who are racists, how 
they got by the Board of Examiners is hard to tell; but they did; and 
I meet them. But, the rest of them are not. They go into e sc o 
with the very best of intentions and with real enthusiasm; but they 
don't have the knowledge and they don't have the skills and ^cy 
have the tools, and Charley Cogen has pointed out that they re not 
wetting the kiid of help that they need. I think it behooves us to 
ask "Why?" Then, maybe we ought to look at aome of the superv sors 
we've licensed. After all, supervisors are former teachers, and 





whatever inadequacies they have are distributed in the same manner . 
Supervisors forget that they have been promoted from among the teach- 
ers ; they try to put a halo over their heads as though they don*t 
display this equal distribution of bell-shaped competency. They 
think that they are all equally competent. We have to look into 
schools in Harlem, for example, and the Harlems throughout the coun- 
try and lbok at the holding power of faculties. Why is it when you 
have two schools, six blocks apart, one principal has a holding power 
and his seniority range of his faculty is that they stay there 11 to 
lU years? And why is it that the next fellow, every September has 
to start with a brand new faculty? The two schools have the same 
student body, are in the same socio-economic area, same parents, and 
the same curriculum. 

I think that we ought to look into this aspect of the 
problem. What is it that makes one person successful in this area 
and the next fellow unsuccessful? When the teacher gets into this 
situation, not only doesn’t he get the help he needs, but he probably 
has a class that is difficult because of patterns of behavior result- 
ing from retardation that already exists. The incidents of disci- 
pline are higher in this area and the teacher, therefore, falls into 
one or another category. He either throws in the sponage and says, 
"I've had it" after a short time, or he takes a look at the school 
and doesn't even show on the first Monday. Some of them will go on 
the Monday, stay Monday and Tuesday, and they never reappear on 
Wednesday. Some of them are more adaptable; they figure out a peda- 
gogic device by which they can somehow or other control the disci- 
pline by the use of some sort of materials; but, it's not a true 
learning situation. The kids are managing not to kick up and then we 
wonder why, at the end of the year, there's reading retardation. The 
reading retardation is there because there has been schooling, but no 
learning during that entire year. 

There is the really professional teacher who wants to do a 
good job but he really is inadequate because he has never learned 
the necessary skills from his college training. Nor has he gotten 
it from his principal. It may very well be that the principal doesn't 
know how himself. It may well be that the principal starts with the 
attitude that a lot of these kids are ineducable and that if you just 
hold them in line, "that's the best and the most we can expect from 
you." I've heard this from some principals around town. What I'm 
really asking for here is that we begin to look at the college pro- 
gram; that we begin to look at what's happening in the schools; 
what's happening with respect to the kind of supervision we get. 

We must see a distinction between the "mobile population" 
and the "stable population." It's shameful, but the permanent record 
card has a tremendous story to tell at any time, including the number 
of times a child has moved when he comes to a particular teacher in 



the fifth grade. What has happened to his scores as he stays longer 
in school is that they have leveled off at a plateau at the fou 
grade or thereabouts. We could make use of this data in a _much be - 
ter way if we had the facilities. Unfortunately, our psychologists 

go and^ become members of Boards of Examiners and then *hi s is" 
leges; so, we don't have enough psychologists; but I think this 

an area we could look into. 

What I am saying is that the pathology that Harry Gilbert 
was talking about is present in a much larger percentage of the to- 
tal population than you're willing to conceive. It s a very s 8 

nificant^part of tt/population and it's the Part of the population 

that interferes with the normal learning process of the kids who ar 
not as mobile and who really are stable and would have ordinarily 
done well. I think we ought to determine whether we can deve 
residential centers. Because the question of getting parental con- 
sent is very difficult, as Harry indicated, there ought to be 
kind of program that might be long time day starting a 

o'clock in the morning and returning the child to *is home at 6 
. _ L _ +hp pvpnine There is a tremendous value in this kina oi 

program. Such a setting should have the proper clinical a^a^ments 

to the school and be properly designed to give it * ® f £® ;^ cs 

When vou go into a modern high school and examine the Home Economics 
Department^ you see a lovely apartment. There is absolutely no 
reasonwhy this kind of plan couldn't be built into this kindj^^ 
center It costs money; it costs a lot of money. I 1 . . 

superintendents of schools, chairmen of departments.a^principa 
fighting as hard for more money for the schools as bharley jf g fhev 
has fought and as 

place their emphasis on an index for salaries. 

What I would like to see is an amalgam of the parents who 
are so disgruntled (and rightfully so) since their chiidrenare not 
learning together with the entire professional staff, marching 
gethe^and^ndicating that we are suffering from a very severe social, 
disease which requires fantastic sums of money. Now, I don } Sg*. 
this monev to go into auxiliary, after-school services. I think 
vou 're wasting your time. Uncle Sam appropriates lots of money and 
?r e oes into trying to teach the kid how to read between three and 
four to theafternoon, when for 7, 8 or 9 years of schooling between 
9 and 3 he hasn't learned. This is very nice added income for 
teachers . My membership would be disgruntled if they thought I was 
t^inTtheir extra income away from them; but, educationally, it has 
little value. If you're going to get money, from Uncle Sam, let s 
U it into the kind of day school program that we require^ and no 
waste it in these peripheral auxiliary, after-school activiti , 
which look good on paper, but don't do the job. 

Paul Denn: Thanks a lot. I'd like to get around the 

country. We got as far as Detroit. How about Chicago, Los Angeles, 

Washington? 
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Wayland Baldwin: I*d like to say something about Los 

Angeles . 

Paul Denn: Would you? Not a speech— just some of your 

problems, or ... 

Wayland Baldwin: Frequent references were made to this pro- 

gram as conducted in New York City. I expressed a personal opinion 
to the effect that I would like to see a More Effective Schools pro- 
gram for all boys and girls in Los Angeles and in New York. The pro- 
gram as conducted in New York includes less than 1 per cent of the 
students in disadvantaged areas. With our limited finances we cannot 
implement this type of program without jeopardizing the educational 
program of the rest of the 800,000 boys and girls. 

Paul Denn: How about other areas of the country, Phila- 

delphia, Chicago? 

Richard Sanders: I am Dick Sanders from Chicago, and I 

have been listening with great interest for two days. I see many of 
I our Chicago problems appearing in other cities. May I get back to 

this business of teacher selection and how it relates to what we 
have been talking about here. 

Illinois law, at least as it relates to teacher selection 
in Chicago, is a little like Federal Civil Service as reported here. 
The State Legislature has said, in very few words, that a board of 
three examiners shall examine all teacher candidates in character, 
scholarship, and general fitness, and prepare eligible lists from 
which appointments must be made in the order of merit. 

The statute ends there, so all the rest of the job is ours. 
This is good in many respects because it gives us (the Board of Ex- 
aminers) a great deal of flexibility but it also means what it says. 

We must provide merit lists and appointments must be made in the or- 
der of merit. We could not do as A1 Schiff says they are doing in 
Detroit, because the law would not permit us to do it that way. Our 
personnel department must appoint in the order of merit. 

Perhaps a few figures would give an idea of our problems. 

We have a total educational staff of about 23,000. About 7£ to 80 
per cent of these are regularly appointed teachers who have been ap- 
pointed after taking examinations and appearing on merit lists. This 
leaves us with about 6,000 positions this year that had to be filled 
with teachers holding temporary certificates. To do this and to pro- 
vide qualified substitute teachers, we have issued since July first 
I about 11,000 temporary certificates. We have an over-supply in many 

I areas, and a shortage in a few areas. If it were not for the fact 

I that we oould use our surplus High School English and History teachers 

I in our elementary schools, we would be in trouble. In order to 
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r teachers for our elementary schools and for 

provide more available we have issued several hundred Pro- 
few other areas of sho ^ a f e * , we . oaT1 ril dates who have Bachelor ‘s 

Sr- — - 



T An not think that certification of qualified teachers is 
the r eal probiem in^Chic ag° > ^^p^em^^e shorties of * Editions 

incentives for an intelligent deployment of our staff. 



This problem is very complicated and ■ 
it here, but I do not think that many of us a all 8 t r ied some 

s&risas: r-iffit - 

“« “S 3K SS^«.iTtU ™ “ °‘ 

schools in the inner-city. 



Perhaps I . a little ^^^^1^ 

%£S£» 1 «52trS sTblast^furnace ' or^an S 

CLTe the least^money far away from the heat and the 

hazards. 



, * t Viai-iave in ” combat pay” and may- 

Now, I am not saying that I b ®^® v ® £oP the same time 

be it is impossible in our business g could give more 

worked, but in Chicago and *LL-city. We 

annual pay to people who ar ,, n f wor ^ a ud thirteen school 

could guarantee them twelve ? ^1 months ^ e ^ k v ^ tion . There are 

months of pay with one school month ( suggested, some of which 

other things that ^conditions such as 

would involve improving the working ciima 

reducing class size. 



I would emphasize that I do not not\T 

in Chicago and perhaps other large citi J ± our abi iity 

It a critical shortage of teachers. willingness to pay 

to reduce the heat »J U» • iTonfway^r another, to 

the people the amount of money necessary, 
get ttem to do the Job that we need done. 



. * « t -p-vnd T want to make another 

As long as I have the *1° * here i n 0 ur discussions. 

nrc; s«s ,j. '»« s"s“s 

S tw— • *" - 01W ° f alc “" 
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The law says that the Board of Examiners must examine character, 
scholarship! and general fitness of all candidates to teach and 
placfsuccessful candidates on eligible lists in the order of merit. 

The Examiner 1 s function ends right there. Appo « . . 

of meXand intelligent deployment of staff is a personnel function. 

Sam Brownell was saying that big city problems are unique, 
and I am sure they are, at least in connection with teacher selec- 
H —a staffing I would agree with him that in a market of a 
g^at^hoftfge !!' teachers, our main job is the elimination of poor 
risks. On the other hand, in areas where we have a surplus 1 * 

teachers, our job will continue to be the selecting of the 
cording to our needs . 

It should be made clear’ that Examiners are selecting poten- 
tial teachers and not presenting full fledged professionals. It is 
toe fSon oflehool administers to make them real professionals 
during the probationary period or to decide that they have fail® • 
Much^f what we have been saying at this meeting is concerned wi 
this second level of teacher selection involving in-service training 

and in-service evaluation. 

Paul Derm: I think the nation's capitol ought to have 

something to say here. 

Rufus Browning: I think we have a problem that is dif- 

fereut from thLe yThlve been describing. We work under Confess, 
nnd dod bless our Congress because last week it raised my pay and I 
Sfiu a State of euphLia. The teachers' salary plan and everybody's 
salary except the Superintendent and his single deputy, went up P 
oroxJmately 6 ^ percent and we needed it. President Johnson signed 
toe ^lai-y bill a week ago. The week before that when he got back 
from Southeast Asia, he signed our budget. You may ^ink that 
+ . Vmrtoct. for next years it is notl It is the budget that s sup 
^ ed to sWt tofs past July first. Since July ^st we've been 
^der the same budget as the one from last year. Now in the middle 
of October S November we have the job of finding 5lS new teachers, 
including^ounselors and librarians. Now that to us is a sizeable 

task • 

Problem number two: Washington, D. C.Negroesnowcon- 

Ptltute 91 per cent of toe student body. This and the fact that 

most of toe whites are congregated west of Rock oJf^As 

an almost impossible situation to have classroom integration. As 
far as staff integration is concerned, we do have a lot of it, 
the integration is very imperfect. I personally am 
the inteeration of staff, but as a personnel man, I have little con 
trol ovefassign-ments! Two out of three of our teachers are Negro as 
are 1 two out of 8 three of our administrators and 

having a great deal of difficulty in locating highly qualified white 
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individuals to become teachers. Our white people, either through the 
fact that they are disappearing in our system or they dislike the 
situation, are now applying for supervisory positions, principalships , 

etc. 

One of the problems which has been partially solved con- 
cerns our procedures for employing teachers. The Board of Education 
recently changed our teacher employment rules, which makes it pos- 
sible for us to better compete with our blue-ribbon, silk stocking 
suburbs. We* re surrounded by such suburbs and they have approxi- 
mately twice as many people as we have in the city. As you know, the 
boundaries of our city never expanded. We have now changed our rules 
so that we can take advantage of what is perhaps our greatest at- 
traction, that is being in the shadow of the Capitol dome. We prob- 
ably have more good walk-in candidates for teaching positions than 
any city of comparable size. When the bureaucrats come to Washington, 
very often their wives want to teach. The wives of officers, sci- 
entists, etc., also provide us with a fine pool of talent. Under our 
old employment rules, which were very wonderful for the depression 
years, we were too slow to process teacher applicants and the people 
went to Arlington County, Virginia, Montgomery County, Maryland, or 
somewhere else. We now have rules which permit us to compete with 
the suburbs and our salary schedule is the best in the area. Of 
course, our salary advantage will only last for about one year, be- 
cause the suburbs are already acting to meet our current teacher 
salary schedule. 

Paul Denn: Well, thank you very much. You've been an aw-„ 

fully good group; you've participated, and I've been a poor chairman 
because you participated all along and I let you run much too much 
overtime . 
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Session No. 7 5 Needed Research in the Area of Teacher 

Selection 

Chairman: Murray Rockowitz, Board of Examiners 

John C. Flanagan, American Institutes for Research in 
the Behavioral Sciences 

Harry B. Gilbert, Pennsylvania State University 



Paul Denn: Again, now I begin to feel that our conference 

is a success because when I see various members caucusing before 
they come in to the plenary session, it looks as if we've had some 
effect on some of the thinking on the part of members of the group. 
This gives me an opportunity to express the thanks of our Board and 
of the group as a whole to the powers that made it possible for us 
to meet, the U.S. Office of Education, represented by Dr. Steiner. 
We'd like him to stand up. (Applause) Thank you very much. Dr. 
Steiner is the Associate Research Coordinator, in charge of Ele- 
mentary and Secondary Research in the U.S. Office of Education which 
has made this conference possible and which I trust will make other 
conferences of a similar nature and on similar problems possible in 
the future. I'd like also to express my deep regret that Dr. Bogen, 
the Chairman of our Board, was not able to be present at these meet- 
ings. I think I told you when the conference first began he suf- 
fered a very severe knee injury, a torn cartilage, I believe, and 
had to undergo an operation. We understand that the operation was 
successful, and he's resting comfortably. I think he would have 
been much happier, much happier than I, I think, here at the lectern, 
but I'm sorry that he could not attend. 

I also want to state my regrets that, not realizing that I 
was going to be thrust in this position, I made some other commit- 
ments and I plan to leave this morning. So I'd like to take this 
opportunity to thank all of you for cooperating so nicely and so ef- 
fectively and making this job that much easier for me. And while 
I'm in the mood to thank, I'd like to thank the members of our Board 
and the research associates who did the planning, did all the prepa- 
ration, again who made this an easier task all around: Dr. Bogen, 

Dr. Gilbert, Dr. Perry Kalick and Dr. Gerhard Lang, and of course, 
Mrs. Louise Klein. 

Finally, I'd like to thank all my colleagues on the Board 
of Examiners , each of whom had a specific task to carry out in the 
course of the conference, chairing various panels. Much as I dis- 
like to say it, there are a considerable number of benefits that 
arise from a war crisis. Research, for example, is carried out under 
pressures of war much more effectively, much more rapidly than in or- 
dinary peace times, and the finances for such research miraculously 





become available. Host of this research we know is in the biological 
and physical sciences . But I recall that in the course of World War 
II there was a considerable amount of research in the development of 
personnel instruments. The assessment of personnel, for example, was 
an important factor, the work of the OSS, and of various war agencies 
in England and elsewhere. Perhaps with the war on poverty some more 
worthwhile needed research, particularly in the area of education 
where we know it is badly needed, will result. And this morning* s 
panel is devoted to the problem of where to place that research. I'd 
like to turn the meeting over at this time to my colleague, Dr. 

Hurray Rockowitz. 

Hurray Rockowitz: Thank you, Paul. For me this conference 

has been an intense personal experience. As the junior member of the 
Board of Examiners, I've seen a lot of names become people, and in 
the research field alone, for example, I used to read the studies of 
David Ryans. Until now, he was a name and last night he became a 
living being for me, and the legendary figures like Brownell and 
others have become part of an environment of which I am a part and, 
therefore, I can say that this has been for me a very exciting ex- 
perience— that of sharing the thinking of these very considerable 
individuals . 

If we were to look at this entire conference as a develop- 
mental lesson, we are in the position of giving out homework assign- 
ments this morning. We're going to tell you and I say we, the panel- 
ists principally, are going to tell you what you have yet to do in 
terms of further study in order to effect the objectives that we all 
came here to discuss. 

Dr. Schiff was much surprised by the budget of the New York 
City Board of Examiners— 1-1/2 million dollars per annum. And yet, 
we have here with us the two research associates that the Board has 
had in the last few years, and we never could get them together at 
one time because of budgetary limitations. The maximum research 
staff that the Board had available to it to evaluate this 1-1/2 
million dollar yearly operation was one research associate. And even 
as good as each was individually, each year I think we'd agree that 
this allowance was inadequate. A few brief words with regard to the 
functions of research. 

Last night I just listened to a number of statements and I 
heard all kinds of assumptions being made. For example, an assump- 
tion was made that a school could not be staffed because of the pe- 
culiar nature of the principal, alone. Another assumption that was 
made was that empathy between teacher and student is more easily at- 
tained if there's a close match— social class of teacher with social 
class of student, race of teacher with race of student— we can go 
on with religion, national origin, etc. These are the types of as- 
sumptions that have to be tested in the crucible of research. 
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I was told that it's nice to inject some levity into the 
proceedings. I can advert to Oscar Wilde's classification of lies* 
He says there are ordinary lies, white lies, and statistics. Re- 
search can be perverted, and it has been. Hitler had his research 
staff, and Garrett is considered a research person too. Research 
has been perverted to national interests so that the Soviets dis- 
covered baseball, but what we're talking about now is the type of 
research which seeks the answers to persistent questions. I think 
that what came out of yesterday's discussions was a general feeling 
that there's a great deal that remains to be done. Dr. Bray indi- 
cated that we hadn't even gotten to the problem of establishing 
criteria, and Dr. Ryans stressed that the criterion problem was 
still primary in his thoughts after a few decades of work in the 
field . 



This morning, we are very privileged to have with us two 
gentlemen who have extensive backgrounds in research. Dr. John C. 
Flanagan fills in the gap which was left by Dr. Bray yesterday morn- 
ing. He's a Harvard man and he received his doctorate from Harvard 
in mental measurement and went on to extensive experience with the 
cooperative testing service of the American Council of Education. 

He did memorable work in the Air Force in selection of personnel and 
for the past 20-odd years he has been connected with the American 
Institutes for Research which he founded and of which he is the 
director. His most extensive current project, the one which is be- 
ing watched with considerable interest, is, of course. Project 
Talent. I feel very privileged to present to you now Dr. John 
Flanagan. 



John Flanagan: I have been very interested in getting back 

in touch with the field of teacher selection after many years. In 
1939, at the suggestion of Ben Wood, I wrote a proposal to get the 
National Teacher Examinations funded by the Carnegie Corporation. 

Dr. David G. Ryans was selected to undertake the direction of this 
program. Since then some progress has been made but as Dave and the 
other people who are working most intensively on this problem indi- 
cated yesterday, they have hardly scratched the surface in the past 
25 years and it is certainly time for a real major effort. I'm very 
encouraged about the likelihood that a major effort will be made on 
the basis of what I heard yesterday, and it seems to me there are 
problems. We have difficulties, but they do not represent insur- 
mountable problems. 

I can recall a group of researchers sitting at lunch in the 
pentagon and saying, "What we ought to do now, right in the middle of 
World War II, is to bring in an applicant group within which we don't 
do any selection and see what happens. We'd find out whether our pro 
cedures are valid. Nobody will believe now that we're only selecting 
the top-scoring applicants for training, that the others would do as 
poorly as we say. They'll say we don't really know what the other 
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4 .+^n +r> He said he felt it was a waste of tine to prepare 

such a proposal. So I sat down and started to *ri te j 

and the more I wrote the more reasonable it seemed. By th 
S tooITgh! I bought it sounded so sensible they wouldn't turn it 

down, and they didn't. 

They let us bring in 1,000 applicants. All the applicant 
had to say^was, "I want to be an Aviation Cadet," and he became an 
Aviation Cadet, no matter what he scored on the screening tests. 

They all wanted to be pilots, so they became pilots regardless of 
theirscores on the classification battery. This gave us a real pic- 
ture of just how effective our procedures were. 

A situation which is more analagous to the one you have at 
the present time developed after the war. It became quite diffi ^ 
to find aviation cadets and the personnel people said, we 

ford to screen and classify these people the way we . sueees- 

just have to take what we can get." So we c^ up wm the sugges 
to continue to give the tests but accept all of the otherwis 
Qualified applicants to determine how the procedures were working in 
TOace time. PP The statement was already being made that tests jay P " 
vide an effective screening device, but in peace t±m ^ Generalize 

Z described 

yes terday . Thu s , we had a very fine opportunity to again find that 
the procedures that were being used were very effective. 

These procedures were developed using research methods. 

»fter listening to the discussion yesterday, it appears that a g 
^ll of resS is needed in this field and I'd like to run over , 

^er four hidings, the kinds °f research that I think^ugM to be 

first, determination of the patterns of teacher ®’ s 

mmm -Ms? 

selection. 

The first of these is research on determining the behaviors 

“.SEE Mentation 
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are that instead of a single set of specifications for the require- 
ments of an effective teacher, there should be a very large number 
of sets of requirements defining the behaviors which make for effec- 
tive performance of each of the various aspects of the teaching task. 
Although such an analytical approach has advantages, it becomes of 
great value in a situation in which the activities of the teacher 
are likely to change markedly with the advent of new teaching meth- 
ods, new objectives, new technology, and the provision of clerical 
assistance by computers and teaching aids. 

Effective teaching in the past has involved many activities 
such as lecturing, maintaining discipline, leading discussions, 
checking on students' learning, correcting students' errors, stimu- 
lating students' thinking and arousing students' interests. Although 
many of these activities are likely to be important for effective 
teaching in the schools of the future, it seems probable that there 
will be considerable change in emphasis. More importance will be 
given such activities as individual tutoring, assisting students to 
plan their educational program, developing a sense of responsibility 
in the student for his own educational development, helping him to 
discover his interests and special abilities, helping him to discover 
the satisfactions that can be obtained from creative activities and 
assisting him generally in his personal and social development. 

Since it's unlikely that most of us will be able to predict 
the precise nature of the teacher's role in activities in the school 
of the future, it is of considerable importance that we carry out re- 
search to define the requirements for effective teaching of many 
kinds. It is also important that in gathering this information, the 
characteristics of the individual student involved in the teaching 
activity be clearly specified. It seems likely that the require- 
ments for motivating one type of student are likely to be different 
from those for motivating another. 

In carrying out research in this field, it is proposed that 
task analysis procedures and the critical incident technique be used. 
These can be supplemented by other observational analysis procedures. 
A device which has already shown its value in this type of research 
is a video camera which can capture the teaching situation and play 
it back for observation and analysis as many times as is required. 

The procedures of task analysis are likely to be of special value in 
developing requirements for new types of teacher activities. Of 
course, in using this procedure the characteristics of the student, 
his previous knowledge and many other factors can be specified and 
varied at will. 

The critical incident technique also makes possible the de- 
tailed description of the specific student and the definition of the 
particular aim of the teacher's activities. It should be of great 
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value to collect many thousands of incidents, each of which is clas- 
sified according to the type of student and the specific teacher 

activity involved. 

Before we leave this matter of teacher requirements and 
the study of teacher requirements involving observation mentions 
some of yesterday* s presentations, I should say that perhaps the 
study of teaching as it*s going on now will be fairly inadequate. I 
recall an instance which occurred in carrying out a study for a 
corporation of third-level managers. We did a very careful study ^ s " 
ing task analysis procedures; we had somebody watch and record what 
they were doing every minute, every $ minutes and for a whole ay. 
Since a number of people were in this position, we obtained several 
thousand critical incidents. We gave this company a very good pic- 
ture of just what their people were doing. This was to have been 
the prelude for selection, but when they looked at our report -and saw 
what their people were doing with their time, they said, »0h no! We 
don*t want any people selected to do that job. As our first jo , 
we* re going to change the nature of this job." 

We might find a similar set of circumstances if we went in 
and found out what our teachers are doing every five minutes by us- 
ing a time sampling. We might very well say, ' >'We donjt want to se- 
lect teachers for that job; let's change the job." I«m suggesting 
we get requirements for each task in the job, then we can c ang 
emphasis without major change. 

The second area of need in research is the evaluation of 
the effectiveness of a teacher. Of course, in this case also the 
focus would be on the effectiveness of the teacher in each of a wide 
variety of activities with each of a number of types of students. 
Teacher effectiveness may be evaluated either in terms or the process 
or the product j as: assessment procedures are developed which measure 
all of the desired changes in student behavior. The computers pro- 
vide efficient storage and analytical procedures for these a a; 
seems likely that much more detailed records can be collected and 
analyzed thL has been characteristic of past evaluation procedures. 
It appears that if individualized educational development procedures 
are used, we»d set specific attainable tasks for each student, the 
details of whether a particular teacher or student attain the goal 
sDecified will provide a much more precise record of teacher eiiec- 
tiveness than has been previously available. High priority should 
be given to research and development work on procedures for a com- 
prehensive evaluation of the teachers* effectiveness in many activi- 
ties with many types of students based on systematic measurement of 
the effectiveness of the instruction. 

It also seems desirable to continue research on process 

variables in teaching. It is suggested that the ™? st P™£®^ H" 6 
of research on process variables involves systematically developed 
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observational record forms on which specific observations by a com- 
petent judge are made of effective and ineffective performance on 
the part of a teacher. These, of course, should be related to the 
particular activity and the particular type of student involved. 

A factor which could have considerable influence on the 
quality o: teachers is the use of merit increases and the assignment 
of tenure status only as a result of demonstrated effectiveness. 

This was mentioned yesterday. The lock step promotions characteris- 
tic of many school systems and the failure to eliminate the ineffec- 
tive teachers produce a situation attractive only to the mediocre 
and marginal teachers. Of course, improvement in these areas must 
be based on valid procedures for the evaluation of teachers . Re- 
search on the evaluation of teacher effectiveness should be given 
much attention for another reason which is its essential importance 
to othar types of research on teacher selection. 

I should perhaps underline at this point the difficulty of 
product evaluation of teacher effectiveness. In many studies over 
quite a few decades now it has been shown that within the variation 1 

in quality of teachers that we find in the United States, the dif- j 

ference in the amount the students learn as a result of being taught I ] 

by one of the better teachers rather than by one of the poor teachers j 

is relatively slight. It is only a few per cent. j 

When we're talking about the problem of what effect the 
teacher has, we have a situation in which in the total variability 
in students' performance only a small proportion of it is under the 
control of the teacher and can be attributed to variation in teach- 
ers' effectiveness. From one point of view this can be regarded as 
a good thing, since even if we don't have very good teachers the 
students will learn anyway. Although this may be the case in many 
instances, it does represent and provide problems when we are trying 
to do research in this area. 

The third area of research relates to the development of 
predictors of effective teaching. It appears that long range follow- 
up studies are among the most important approaches to this problem. 

It is anticipated that data from students tested as twelfth graders 
in Project Talent who have graduated from college and entered teach- 
ing will become available this fall. Similarly in the next three 
years data from other classes will be collected and analyzed. These 
data should be of distinct value in defining the type of individual 
who enters a teaching career. Of even greater value will be the ten- 
year follow-up plan for Project Talent which will begin in 1970 * 

These studies will provide data on those who have, insofar as we can 
measure it, not only selected careers in teaching but have been 
found to be effective teachers over a substantial period of service. 
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As in the case of the other types of research studies men- 
tioned, it seems important that predictors not be focused °nthe 
quality of teachers in general but rather the effectiveness of the 
teacher in performing specific roles and activities. It is also 
lieved that the trend in research with respect to predictors w e 
inthe direction of a very large number of specific Pjred^ors rather, 
than a few more general measures. Drs. Bray, Baker, Med y 
Si emphasized the need for supplementing pencil-and-paper aptitude 
types of tests with other types of information onthe teacher. I 
ni gh t say that as an old line measurement Person I thought some of 
them overemphasized the need for supplementing these 1 

several recent AIR studies it has been found that pencil-and-pap 
validities could be increased by situational performance tests and 
by different kinds of observation of performance in a realistic situ- 
ation I'm sure that a procedure which does not include such ob 
servation of behavior in relatively realistic teaching situations is 
going to be less efficient than one which does have tested and vali 

dated measures of this type. 

The last area of needed research relates to the conditions 
of teaching with special emphasis on those which have negative selec- 
tive value. It is well known that the turnover in teaching is ve y 
high and it seems important to determine why teaching loses some of 
the most effective individuals. One of the problems calling for re- 
search in this area is the effects of family duties on remaining in 
teaching. It would seem worthwhile to study the question of whether 
p^t-iime employment or something less than full-time employment 
might retain some of the better women teachers. 

Other research topics in this area relate to thestudyof 
specific teacher activities particularly with a view to determining 
whether or not some of the clerical and administrative duties could 
be performed by teacher aides, clerks, or computers. 

Another valuable study might be directed at the question of 
vhv many promising young teachers leave teaching early in their ca- 

«er^ ifseems likely that the administrative practiceofassigning 
new teachers to difficult classes, loading them down with the more un- 
pleasant extracurricular assignments and similar procedures which gi 
the new teacher a bad initial impression of teaching are responsible 
for losing effective teachers. 

In conclusion, it is believed that the new methods of teach- 
ing will make it possible to evaluate teaching effectiveness much more 
precisely than has been the case in recent years. This will “a*® re- 
search on the characteristics for effective performance in specific 
teaching tasks both feasible and important. As proposals for newedu- 
cational programs are developed, they should be accompanied by plans 
for research^on the teacher's role and activities in the new program. 
Per haps one of the greatest needs for research in teaching is the 
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application of systems concepts to the teaching and learning system. 
This application would include comprehensive measures of the in-put 
and out-put of the system as well as study the process variables. 

Fbr such studies to be effective, much better measures of the in-put 
and out-put of the system in terms of assessing the student's be- 
havior concepts and abilities will be required than are now available, 
^rhaps the first step in a program of teacher selection should be 
the definition and development of measures of the products of teach- 
ing. Unless we have valid measures of the effectiveness of a teach- 
er's efforts, other types of research nu3t be regarded as tentative, 
exploratory and perhaps in many cases trivial. 



large order and a quite complex homework assignment for us, but I 
think we welcomed your considered remarks. 



shoes I'm trying to fill. I inherited Harry Gilbert's responsibili- 
ties on the Board of Examiners, and aside from a deep personal debt 
to him which I publicly acknowledge, I am finding out that the job 
he did, the image he created and just about everything that he as- 
sociated with the job has created a kind of halo effect for me. He 
wears two hats at the present time: he's still a member of the Board 
of ^ 'aminers but he's also Assistant Dean for Resident Instruction at 
the College of Education of the Pennsylvania State University. In 
New York City, he served as a teacher of children in classes for 
children with retarded mental development, he was a school psycholo- 
gist and a supervisor of school psychologists before becoming an 
examiner. With typical tongue in cheek, he likes to refer to himself 
as a clinical psychologist in sheepskin clothes. Dr. Gilbert will 
now lead us further along the path of needed research in teacher 
selection. 



line the specific proposals I shall outline. Permit me to make them 
explicit. One, interest in the area of teacher selection is minimal, 
based upon the actual amount of research under way. However, a great 
deal of interest does exist among teacher personnel selectors and uni- 
versities. The problem is to make patent what's latent. Two, pro- 
fessional teacher selection practices are rarely employed. In large 
school systems that are presumed to be using selection techniques, 
screening is actually done. In smaller, affluent school districts, 
hunch rejections and global perusals sometimes in actual observations, 
serve as selection techniques. A moment's reflection on the current 
shortage of teachers particularly in those school districts perceived 
to be tough wi? n ~ — n 4 *— 4 ‘ u “ 4 ' ■»«= 

warranted . 

I wai 

cate that I th: 



Murray Rockowitz: Thank you Dr. Flanagan, that was a very 



Our next panelist is a gentleman whose very considerable 



Harry Gilbert: There are three basic assumptions that under- 
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was mentioned here earlier, and I'm particularly indebted to Medley 
for the use of the achievement test concept vs. the aptitude test 
concept. There is no need to be bashful or abashed about our using 
this concept of expecting a certain degree of achievement for pro- 
spective teachers. This brought to mind a very recent experience. I 
was walking through a school in a city I shall not name and it hap- 
pened to be on a day when there was a city-wide testing program; one 
of the teachers had put a sign on the door, "Testing, do not distrub." 
As I walked through with the principal, I laughed about it and made 
some comment to the principal that this is one of the occupational 
hazards, when you teach children with reversals, etc., and he said, 
"No, this teacher can't get a regular license; she fails the test for 
written English." It isn't funny, and those of us who use tests of 
written English recognize that a fair number of college graduates 
cannot demonstrate that they can write acceptable English. So I'm 
not at all disturbed about this achievement test model, I think it's 
a good one. We can probably refer to it as content validity if we 
wish, but nobody is going to take away the responsibility that se- 
lectors will have for making a judgment about some expected achieve- 
ment a prospective teacher will have to be able to demonstrate. 

Now the third of my assumptions is this: Since the field 
of teacher selection is a great big area, up for grabs, it is most 
desirable that research be encouraged in varieties of approaches, 
without too much specificity. It follows, of course, that wide dis- 
semination of research be encouraged, and that investigators be 
supported with the notion that hypotheses may be rejected as well as 
verified by experimental data. Regrettably, this simple dictum, 
readily understood by researchers, seems to be a heretic notion in 
an age when innovations are publicized as successes before evaluation. 

Again on this notion of encouraging a variety of . approaches , 
John Flanagan's being here reminds me that he headed the Air Force 
Selection Program, and I worked in the Naval Aviation Selection Pro- 
gram (John Jenkins headed my group). Here were two very competent 
psychologists who started out with very similar problems and came up 
with very dissimilar answers. 



The Air Force approach included a 1 ^e number of psycho- 
motor tests. I went through one of their testing stations at San 
Antonio and it was like a New Yorker's conception of a psychologist 
gone wild. There were all kinds of intricate psychomotor tests. The 
Navy, by way of contrast, came up with a fairly short series of paper 
and pencil tests, a biographical inventory, a short intelligence test, 
Bennett's test of mechanical comprehension. And yet when you looked 
at the results of their validity studies, I can see John's stanines 
now and the relatively smaller degree of failures in training as you 
go up the stanine ladder. A very impressive example of the use of 
their tests, but the Navy came up with the very same thing, and the 
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results were almost the same as you went up the ABCD categories that 
were used in the Navy. I*m commenting here that starting with the 
very same approach , the same problem, two utterly different approaches 
resulted, and' they gave about the same results as fair as I could see. 

Now for the proposals. One, attention must be paid to the 
process of stimulating attention to the methodology of teacher selec- 
tion and to research, which must be inevitably an assessment of the 
validity of presently employed procedures and those yet to be tried 
or designed . Surely we can agree that fundamental research on teacher 
behavior and on pupil- teacher interaction as an outcome of the teacher 
stimulus must be encouraged. What is needed is understanding of why 
so important a field for research continues to be neglected, despite 
promising starts by such as Withall, Mitzel and Ryans. The simple 
answer, too difficult, seems not enough in an age when complex prob- 
lems are tackled with the resources made available by contemporary 
cash flow. I suggest, therefore, that social psychologists and soci- 
ologists be invited to investigate broadly the field of research in 
teacher selection for their aid in understanding avoidance attitudes 
of potential research workers. There is no disposition to hide the 
obvious, that I have made a value judgment and that the purpose would 
be to be able to persuade research talent into action in this sphere. 

I propose, too, that we evaluate the effectiveness of the design of 
this conference with its influential participants as a means of stir- 
ring interest and action in teacher selection practice and research. 

Two, we should take advantage of the current supply and de- 
mand in the area of teacher selection. Specifically, I refer to the 
shortage of teachers as the country begins to wake up to the great 
big need for teachers and pumps green blood into local school systems 
from the great big Federal artery. Teacher shortage gives the person- 
nel research man a rare opportunity. He can lower his selection cut- 
off scores to provide a greater range for assessment of predicted 
validity. It would take some degree of courage to do this, although 
some reflection on the lack of predictive validity data should be en- 
couragement enough. Now this is a point that Mitzel made earlier. 

At the same time, there is always an over-supply of applicants for 
supervisory or administrative positions in education. This should 
serve to be a source of comfort for those who would argue that one 
cannot be selective or try to be, with teachers when we need all who 
apply. Furthermore, it is palatable to applicants to be fed a variety 
of selection techniques when there are more applicants than positions 
to be filled. This presents the opportunity to employ varied and mul- 
tiple devices toward a socially accepting, even willing group. Spe- 
cifically, I»m suggesting that we ought to throw a good deal of energy 
into the selection of supervisors where we do always have an over- 
supply. 

Three, I now come to some specific proposals. First, I 
make reference to the sixteen suggestions for further research on 
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pages £5 to £7 of "Teacher Selection Policies and Proc? ares in Large 
School Systems in the United States." That is the first study that 
we completed in this series. There should be enough ideas here to 
generate several bushels of Ph.D.'s or Ed.D.'s. I should like to 
take the time to express myself on the perennial dissuader, the lack 
of sound criteria. The fact is that sound criteria exist only in 
relatively simple occupations and not in complex professions. Worker 
output in terms of quantity or quality or time or a combination there- 
of can be employed as criteria for production line employees, but 
everybody knows how hard it is to define "success in teaching," par- 
ticularly since this amorphous generalization, bad to begin with, 
keeps shifting in different school settings at different levels and 
with different subjects. At least we have learned from previous 
study that the overall general estimate of "success in teaching" is 
a concept to be discarded. We can work with dimensions of teacher 
performance as the local hiring school systems define them. This is 
important; these dimensions are going to be defined differently by 
people who hire them, but I think Harold Mitzel made this point 
earlier. These dimensions may be with respect to teacher behavior 
such as "evocative of pupil participation," "encouraging pupil-pupil 
interaction," "accepting of deviant behavior," etc. They may be 
with respect to the ultimate in expectancy, namely, pupil development 
in skills, knowledge or attitudes. Let's not beat a dead horse. We 
can all agree that research is necessary to refine and to define the 
dimensions and to determine methods objectively or reasonably objec- 
tively to assess them. I propose that we accept crude estimates of 
the extremes in dimensions and not wait for the millennium. I pro- 
pose that we engage task forces of teachers, teacher educators, 
school administrators and personnel experts to agree on working 
definitions of dimensions of teacher behavior on a scale from most de- 
sirable to least desirable. Then obtain nominations for extremes on 
these scales, the best and the worst. Various procedures can be em- 
ployed for these nominations. They can be peer nominating techniques, 
observer ratings, supervisor ratings, pupil nominations and measured 
pupil performance. Hopefully, a variety of techniques would be em- 
ployed as Doug Bray suggested earlier. I am arguing in short for our 
getting started with best available techniques to obtain criteria 
while others among us continue more basic research. 



mmm 



In another setting at the A.P.A. annual meeting in Septem- 
ber, I also urged that we utilize the best informed personnel minds 
to review present selection procedures. While I would hardly clas- 
sify such activity as research, I do maintain that some improvement 
in present procedures can ensue without waiting for results of years 
of study. In short, I am urging a redirection of energy to the 
problem of teacher selection, research and procedure, in a reasonable 
and practical way without sacrificing the on-going need for basic 
research. 







Finally, I would like to propose the need for establishing 
a clearing house for research and practice in teacher selection. I 
don't know who would establish this, I believe the field is importan 
enough that some university interested in developing this branch of 
educational administration might be encouraged to assume such leader' 
ship. Certainly a repository of findings with dissemination facili- 
ties would be a major resource in the development of interest in the 

field . 



Murray Rockowitz: As is frequently the case, Dr. Gilbert 

is heard as the honest broker to bring together the challenge hurled 
by Drs. Bray, Ryans, Mitzel, and Medley. In any case, I think that 
we're at a point now where questions can be fired at the panel from 
you to clarify any thing that they have said this morning. 

David Darland : Aren't we really talking about applied re- 

search? Would it be feasible for some university, or combination of 
universities, to define this problem in all of its ramifications so 
that the research would be directed toward a team kind of an effort 
to get away from our tendency to be so piecemeal? 

Paul Denn: As I get it, Dr. Darland 's proposal is not to 

have the clearing house of results of research, but also a clearing 
house prior to research. 

David Darland: I'm suggesting an approach for getting at a 

problem of massive dimensions with appropriate means. 



approach has not been very good. Researchers typically just don't 
want to do somebody else*s research that they have defined. I think 
that in certain instances people have set back the field ten years by 
outlining all the research and spelling out the research designs for 
all the projects that needed to be done in that field and, of course, 
this immediately then took all of those out of the realm of possi- 
bility for any self-respecting independent researcher, because he 
wouldn't do a study that had already been designed and printed in a 
book by another psychologist or researcher. So that I don't know, I 
think that theoretically it sounds fine, actually I'm not so sure 
that that's the way to get things done. I guess that the particular 
formula which I have found very effective is that instead of piece- 
meal projects, you have a program and you could have a program and 
perhaps that's what you meant. It's not sort Of a committee where we 
all get together, but you have to have a director and you have to 
have somebody planning the program, or a group planning the program, 
and then make assignments of things to do. 



Discussion 



Murray Rockowitz: Any comment on that? 

John Flanagan: My experience with having sort of a committee 
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David Darland: For the record, by God, I didn't use the 

word coamittee. 

John Flanagan: All right. Well, I think there* s no ques- 

tion but what one needs large scale, long range, concerted attacks on 
things and I'm certainly on the record for it. A. T. and T. would 
fail if they didn't do that. That is correct, and so would Standard 
Oil. Tes, I think that that's a very good example where in a recent 
study we did for A. T. and T., we probably pushed the field ahead 
about twenty years above where it would have normally been by just 
their giving us enough money to do a good study. 

Murray Rockowitz: Are there any other questions? Tes, 

Dr. Hitioli 



Harold Mitzel: I would like to echo what John said about 

publishing lists of needed research. You know about ten years ago, 
the American Association of College Teachers of Education put out a 
little green book which was called "Needed Research in Teacher Educa 
tion," and as far as I know, not one of the hundreds of ideas or sug 
gestions in that little book was ever picked up. 

David Darland: I want to again insist that I was not sug- 

gesting publishing lists; I was suggesting quite another technique. 

I just want to be clear here about what I was suggesting. 



David Ryans: I just want to respond to Dave here, just for 

the heck of it you know, that he did say couldn't some university or 
some group do this, and I think that's the thing that Dr. Flanagan 
and Harold Mitzel and that I, too, would very much oppose, any one 
group, because there are lots of different approaches. No one of us, 
I hope, is God and knows all the answers of how this should be done 
and there's lots of room for different programs but, if you mean pro- 
gramatic, Dave, fine; if you mean some one group say now here ; s the 
scheme of all that should be done in this area, then I don t believe 

it. 



Samuel Brownell: I would like to take advantage of the op- 

portunity to suggest some things that weren't said this morning in 
reference to looking ahead in this field of teacher selection. As 
I've listened and recognized some of the traumatic experiences that 
we went through when I was in Detroit and we were trying to do a 
better job of teacher selection, I recognize, the change that is tak- 
ing place very rapidly and the changes that are ahead of us. 



In this vein I'd like to comment on a couple of things. 

One is to look at this problem from the standpoint of the person who 
goes through it. We talked pretty much about the students and the 
people who were doing the examining business. I think we have to 
recognize that people who come up to the point where they reach the 
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Board of Examiners or the teacher selection process have put in a 
very sizeable investment over a period of years. Most of them have 
been looking forward to reaching that point. We may well consider 
what is their attitude and what’s the effect on them of getting the 
job or not getting the job. That’s a pretty important decision that's 
made from the standpoint of the human beings that are affected, and I 
think we can’t overlook that. X suspect that their experience in go- 
ing through the selection process has a lot to do with the attitude 
toward the school system based upon whether they are selected or 
whether they are rejected. It has a lot to do, too, with the atti- 
tude of a great many other people who are their friends and relatives 
as to their opinion in regard to the school system, and particularly 
in regard to the examining process. I'm afraid there's been a lot of 
negative attitude developed towards schools and school systems on the 
part of those who have had unfortunate, experiences as a result of 
what has happened to them, and they may have come to respect teaching 
less because of their experiences. I think we ought to realize that 
this is an area which has to be taken into account. 

A second thing that I wanted to comment on is citizen ex- 
pectation of teachers as I see it in our large city schools. I think 
we've centered in our consideration on certain of the aspects of 
teaching. I happened to be unable to be here the night before last 
because I was with a group of citizens who were talking about prob- 
lems of the city and what can be done in the function of schools and 
other agencies toward making cities better places for people to live. 
The concern of the citizens with whom I was meeting was that in our 
approach to urban redevelopment we sometimes forget that cities are 
for people! The question was raised by this group of citizens as to 
whether or not, in view of the complexity of cities, we are going to 
be able to make cities work as places for people to live and to bring 

up families. 

Well, obviously cities have to work, and what's the role of 
the teacher going to be in this? Is there any way that we can in our 
teacher selection procedures aid in selecting those people who believe 
in people and their worth, their dignity, and the role of the school 
in making cities better places for people to live? I'm not at all 
sure but what we may find in the next decade that we'll have more 
citizen pressure on us to have the schools contribute more to the 
human aspect of education and living, than perhaps on some of the 
concerns we have about reading and writing. I think we ought to be 
looking to see whether or not there is anything we can do in our 
teacher selection to encourage more of those people who have atti- 
tudes of responsibility for pupils as important, developing individu 
als, and discourage people getting into teaching who may ^tech- 
nicians, but who themselves are fleeing from their responsibilities 
of making the city a good place to live. 



223 




m 









Now one or two other things that I wanted tdcommenton 
.. t T thl ^ ’ have to work on in- teacher selection. What is the 
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selection procedures as to what the m an *j ln soh0 ol opera- 

teacher orga "J; z t^ 0 pi e S w h o°are in the teaching profession and what 
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some thinking on that. 



T think too, that in this teacher selection area we must 

*» - r “-tssr sxrs 

on the selecting of paraprofessionals or th« 3 aides in we 

they are on teachers. I wouWn- t ^“professional teachers 

have as many people in the school wh ar P teacher se- 

as we have people who are teachers. ***** ^ h ^ ople so that 

SSS s tar- » - “ 

bases where we have to make discrimination. 



Another area which hasn't been mentioned “as to do with the 
rehabilitation 01 tne x-ea-ne , + neoD i e an d then we put 

^intfa^arge 

S£ "tT. a invested d them a. by 

them. 



Finally I'm wondering if this group isn't the appropriate 
group to consider* the matter 

in^ Detroit 'JKZ £ % ^ 
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If the process in Detroit i 8 h should the other city repeat 

ful selection process in L ° s ’ / Wms 0 f trying to find out 
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when they mD J[ e ocess again. And secondly, I think you might 
to go through the process again. » because all of us now 

find out that the procedures work just as well because an oi 
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accept that the children who come from one echool system and go to 
another adjust pretty well. There are some mistakes, but I believe 
this area of reciprocity is one that this group might very well 
consider. * 



Now, what I tried to do then is simply to mention a few of 
tne things that I haven't heard as things this group might very well 
do that I thought might be put in the basket. 



Murray Rockowitz: And we welcome them. Yes, there are 

going to be many comments. 



John Flanagan: I'd like to comment on just one or two of 

these. I think that it's very obvious that the decision should be 
made early, and we've found in our Project Talent study that the guid- 
ance procedures provided students in today's schools are just not ade- 
quate to prepare students to make good decisions in the elementary and 
secondary schools. Of course, the later the decision is made by a 
person to prepare for a career in teaching, the more unlikely it is 
that this person will get the best training for it. In our work at 
the American Institutes for Research at the present time we are try- 
ing to develop a set of guidance procedures based on the Project 
Talent findings which will identify the prospective teacher at an 
ssrly &g6, will help him identify himself as this kind of person, 
and will enable him to plan his educational development. If our 
guidance procedures are correct, these people will have no difficulty 
in obtaining positions in school systems. We need to help people make 
the right decisions early. If you bring college graduates into your 
system and then find out one, two or five years later that they don't 
belong, this is much worse for them and their careers than having 
kept them out in the first place. 

Harry Gilbert: I'd like to make some comments. I'm very 

glad that Dr. Brownell brought in the whole area of the social climate 
in which we are existing. Frankly, it was our failure to get social 
psychologists in here that led us to the conclusion that we had better 
limit our discussions. I think probably the most important thing we 
would do would be to agree that we have to examine the social climate 
which says that the teacher is the most important person in society, 
we entrust our children to him. But the same people who say that are 
not willing to support a teacher in a style to which he should be ac- 
customed. The fact is that in almost any community there is resistance 
to raising teachers' salaries, providing the professional means of be- 
ing a teacher in the real sense of the word. It may well be, as you've 
pointed out, that in the long run, it's the strength of the teacher 
organizations that will provide the professionalization and set re- 
quirements and demands on society that will be the most significant 
thing in this whole field. 
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On a less important level, I "^^^er strongl^a?" 
with the notion of reciprocity ^cause I f define the task for 

it will be the local community that will hav^ ^ aelection 

the teacher in that communi y. expectancies . At the same 

procedures will have to be 8®*^ selection procedures in any one of 
time, anybody who goes through confidence, to get through a 

the large cities ought t0 . ^® ® bl ®> W ^ b i°fis! if we mice it so time, 
selection procedure ® 1 ®?'* e [® - i T i n j act an ordeal. I don't know of 

anywhere that will accept 

r^it^ *• ^ rs ° n for its 

own agency. But that's minor. 



I think the real problem that Dr * 1 '^Lafh^betn 

disturbs all ° f "®- ^here is ™ ° too, and that has to do with the 
said here that I d like to hr g P your own selection procc- 

tendency generally not to wan interview so readily and 

dures. Why is it that people a P L ll Kel i y back around 1955 

rarely question toS a stab It the psycho- 

dynamics °o? interviewing He sugg-ted and l think thatms «_a 

very -rf^s'^io^^nher^rshould get a job or 
cisions about this fel t0 be aslcing of himself. He, 

should not get a job, this is a the notion that he's not capable 
therefore, rejects as unacceptablethenotio psyc hodynamics 
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research. 



Murray Rockowitz: Thank you, ^^^^^^neU ^de!"’ 

ment briefly on °" e P oinb ’^ e u ^“rs i ty r epresentatives here will 
I think that Dr. Gold and the umver y P preparation from 

admit that there has been a pSuoso^hy^ ^ principles 

?™ef l to! t Oold! f in h the°ttach!r preparation courses? Pretty valid, 



yes 



I think that one of fndf- 

study of why the teachers are unwilli ^ P q thgir teaoh ing. 
cated yesterday, to move to the in ^y thls lnn er-city ex- 

What are the motivations that P h nch ^ 0 f course, that the fact 
perience? I suspect, thisisjust a hunch, tQ ’ it is a basio 

JT22 of the programs in student teaching that 
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are centered on the inner-city and the experience in the inner-city 
might dissipate some of the fear that exists because of an inadequate 
contact with the area. 

Jay Greene: One additional area of research I think has 

been suggested, I don’t know whether it’s been indicated specifically, 
might be in the first selection step on the college level. Would-be 
teachers are being selected for teacher education programs in the col- 
leges on various bases . If you compare the inadequacy of some of the 
selection by comparison with the selection that goes into other pro- 
fessions, I think the inadequacy becomes more readily apparent. If 
you want to become a psychologist in a graduate program, there's a 
rigorous selection procedure. If you want to become a physician or 
go into other professions, ditto. If you want to become a teacher, 
there is not that degree of selection. Now some of the research cer- 
tainly should go into helping in that selection and to challenging 
even some of the basic assumptions. It may be that in normal times, 
when supply and demand equalize, that the preparation for teacher 
education should be in the graduate school, after an individual has 
had a chance to develop as an undergraduate. I know the establish- 
ment will oppose that tremendously, but it’s a consideration merely 
for investigation and research as a possibility. If this is what's 
happening in all professions, why isn't it happening in teaching? 

Why isn't it good for teaching? I'm not saying this is the answer, 
but it's an example of challenging assumptions for research. 

The second point is this . Harry Gilbert has indicated that 
to wait for the millennium until we get complete criteria is to post- 
pone action unnecessarily long. He suggests that we try to do the 
best we can with criteria and get along with that job. I would go 
one step further than that. I'm mindful of the fact that research in 
education and in other fields of humanity has very little and long de- 
layed impact on practice. You consider the thousands of research ef- 
forts done in the area of human activity during the year and the. 
infinitesimal effect, unfortunately so, on practices. If you wait 
for research to really lead into making changes, it's a long term ef- 
fort. I think probably that what happens is that individuals doing 
research develop insights into what's happening and gradually partici- 
pate in meetings of this sort, and by discussion bring about changes. 
Therefore, my proposal, Harry, is to go one step beyond and to say to 
prepare to make changes based upon meetings of this sort where people 
who have developed these insights through research, through experi- 
ences, through the cross-fire of exchange, can get together and examine 
practices and suggest areas where different cities, towns and groups, 
can try things out. This is the way changes have been made in educa- 
tion by discussion and the determination often of one person in a com- 
munity to try certain things. I think we ought not shy away from that 
and validation of research findings, but we ought to recognize that. 
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Murray Rockowitz: I'd like to state that this discussion 

will be continued, but now I want to turn the proceedings back to Mr. 
Itenn Before I do, I'd like to say that I'd like to grant a subjec- 
tive" cradeof A to the panel this morning. I think that the homework 
assignment that they've given us is clear. I think 
by the class, that it has certainly been motivated, and it has c 
t.inlv amerced from the discussions that we have been holding. I 
^wfcanlo back to our own bailiwicks with increased insight into 
^afhM to be done and a full knowledge of how little we know About 
tte actual procedures that we have been discussing in the last few 
days. I'll torn this now back to Mr. Denn. 



Paul Denn: Let us take a five-minute break and then re- 

assert^ to consider specific blueprints for action. 
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Session No. 8: Summation of Conference 

Chairman: Harry B. Gilbert, Pennsylvania State University 



Harry Gilbert: Ladies and gentlemen, may I have your atten- 

tion, please. I have a few charges that have been expressed to me. 

I m going to suggest a way of effecting a compromise. Many people 

5 a Iu 4 S i Ug f?? ted ^ ey would like to be out of here earlier than 12:30. 

I think its quite important ; you need time to pack, get ready, etc., 

x-.T f 1 8°ing to suggest a procedure for getting us all out of here 
at 11:U5, one hour from now. 

It is this. It has been impossible to get everybody here 
to participate, much as we would like to have done this. I*ve talked 
this over with a few and it seems acceptable, if you won't object. 

This is the procedure I am now proposing: that we break up into four 

groups of approximately 8 to 10 to meet for 30 minutes, to elect a 
spokesman and to come up with three, four, five specific ideas that 
represent summary, outcomes, and suggestions for keeping momentum for 
this group. There are places right outside in the lobby where you 
can be separated from one another. But you have to limit yourselves 
to exactly 30 minutes, and be back here at 11:15 with an elected 
spokesman who will give a 2- to 3-minute summary of what your group 
has recommended. The spokesmen who come second, third, and fourth 
raist also be disciplined enough to say we agree with Items 1 and 2 
that have been suggested and add their additional ideas. 



Now, does this seem like moving us too fast? I would like 
^ giy e everybody a chance to bring in some suggestion, and I think 
this is one way where we can come up with some specific proposals. 
Does anybody object to this procedure? No? Then I suggest that we 
do the following: Dr. Brown of Los Angeles, will you take the first 

comfortable settee and will Harold Mitzel report to that group. I'm 
going to try to split up some of the research people. Mr. Perz 
will you take a second group, a little bit away from group one/and 
Don Medley, will you report to that group. 

. A1 Schiff, will you take a third group, somewhere further 

up m the corner, and John Flanagan, would you stay with that group. 
And Dr. Sanders, will you stay here with the remaining group. Now, 
what research person have I left out? Dave, will you stay here, 
please, Dave Ryans. It's a nice time to duck under a table, but I 
caught you. Will the industrialists among us break up, be schizo- 
phrenic, be in two places, do anything but, Doug and Paul, don't 
you stay in the same group. And, I think if you just don't sit here, 
and get started but, remember you've got to be back here at ll:l£ 
with an elected spokesman and a 2- to 3-minute summary of sugges- 
tions, specific outcomes, some kind of closure for what this Con- 
ference has meant and what it can mean. 
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out somehow. 



Our Committee, composed of 
P.ul Fitzgerald, John Flanagan, Irving Gold, Harry Miller, Raipn 
WaSe£1?S Viliams and Bernard Berger, believe: 



n ^ there must be a better screening of admission to 

(1) „ TT" . re ,i testing of attitudes and per- 
the colleges of education, a real x-esi^ing 

sonality; 



(2) that the training ^experience the 

college of education level mus believe will lead to more 

effective ^ * ** d " rt * Pid| 



tha + the initiative must be taken and closer co- 
(3) that. t,ne -Luxuxtt t _» an d the teacher train- 

operation must exist between s there's a difference between 

iS instituting, somehow or Other i fetation of large city 

s chool^ys terns ^ Z this fan be worked out mutually; 



(U) that the initial assignment of any teach er^has to_be 

a good running “**£• tTbe°built in as part of the pro- 

tunities for successandthis l>as igors and admin istrators 

under°whom r this" teache^mus tCork , a£ this needs additional research; 



(5) it is important that we have tlachingTbut 

going on, situational performances, perhaps simulated teac ng, 

Sis must be part of the selection procedure, 



,,, +h , rea iiy a dearth of research on "real" teacher 
selection SUS \ \ ^ "SSf- 



(7) there has to be a continuum of 

r » ut'o <• « - 

51 ^ £ ilVoT »re seasoned ^friend teecher, 



teachers for specific jo . herg ^ at teachers will be able to 

flexibility that we want and* trained, but perhaps we have to 

fit in if they're well Qualified and certain ki«is 

rfn a little more than this; in oLner wuiuo, 
of jobs that teachers specifically can do 
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(9) improvement of procedure?. This whole universe of 
the young teacher; here we have thousands of young teachers, "brand 
new spanking" teachers, inexperienced, coming into our classrooms, 
we have a large sample to test and to measure. Let’s see what we 
can do with this vast untapped source. 



And last, but most important, remember, even though all our 
city school systems have said we have immediate problems today o 
staffing the classrooms, let's not forget that we're preparing eac ” 
ers to teach kids in the 70's and 80 's, that we have to research and 
plan, and the role of the teacher for the future, both from the 
school system's and the teacher training institution's point of view 

is most important. 




Harry Gilbert: Thank you, Al. That was exactly three 

minutes, and I appreciate it. Speaking for the next group, Dr. Ted 
Lang, of New York City. 

Theodore Lang: I would like to mention the names of our 

group members first: Mr. Benson, Mr. Cogen, Mr. Entenmann, Dr. 

Gold, Dr. Greene, Dr. Medley, Mr. Perz. 

Going directly to the research that we believe to be es- 
sential, ny committee would like to cite, first, the research in the 
selection of supervisors . We have regarded this area as one of 
prominence because of our feeling that it has been even more neglected 
than the subject of research in the selection of teachers and, more 
importantly, because our observation is that a good principal makes 
possible good staffing. Good leadership in the schools goes far 
toward meeting the staffing problems of the school. . In the area of 
research for the selection of principals, we think immediately of 
the need for establishment of criteria for effectiveness of _ a prin- 
cipal, and second, techniques for the selection of principals. 




Turning now to teachers, my committee feels that the first 
essential is to determine what makes an effective teacher. What are 
the criteria for selection of an effective teacher and what are 
teacher behavior patterns which lead to effective teaching. We be- 
lieve the time to start this investigation is now. Because we feel 
that such research is essential to the improvement of the selection 
process, we urge that at least one large city develop in the near 
future a research program of the kind I have just described. We 
think that this should be a cooperative endeavor of the teacher 
education institutions, the Board of Examiners or the examining group, 
the personnel people and administrators of the school systems and, 
perhaps, of the teacher organizations. In this question of what . 
makes a good teacher, we deem it important to distinguish the varied 
patterns of good teaching in distinct parts of the city or in 1 
ferent subject areas or levels. Picking up what has been said by Dr. 
Schiff, we think that such identification of behavior patterns for 
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specific teaching areas might reflect itself in the selection tech- 
niques, which would become more diagnostic. The results of the test- 
ing of the prospective teacher should then be usable for placement 
purposes. Consequently, we would seek to place in a middle-class 
neighborhood the teacher who could teach well in such a neighborhood, 
and the teacher who has the potential for handling the disadvantaged 
child in the disadvantaged areas of the city, thus leading to better 
utilization of the teacher and enhancing the opportunities for his 
retention. 



This leads me to the next major area of research, the area 
of retention. My committee breaks this into two parts: first, sup- 

port for the beginning teacher; and second, research into the reasons 
why new teachers leave the service. Parenthetically, research in 
the support of the beginning teacher has been started in New Yor.c 
City through the Center for Urban Education, which, in cooperation 
with the Office of Personnel, is trying out six patterns of assist- 
ance for the entering teacher. Research into the reasons why 
teachers leave will be undertaken in New York City in the next few 
weeks. We have already collected data on teachers who left the sys- 
tem during the first six weeks of the term. We shall call them in 
for interviews and find out why they resigned, and also give them 
an opportunity to return to teaching. As Dr. Brownell sugges e , 
we will seek to place them back in the City school system. 

Our committee recognizes the great value of this kind of 
meeting. We think this type of session should proliferate region- 
ally throughout the United States and should be repeated, perhaps 
annually. Further, we urge a discipline in the conduct of future 
meetings, in the sense that each meeting should bring forth a pro- 
gram for the ensuing year, developed by the researchers and the 
practitioners, and that each following gathering should bring a re- 
port of what has transpired in the preceding year, so that we can 
evaluate our own efforts. Finally, we suggest that leadership in 
this entire area might well come through the regional research lab- 
oratories being established under Title IV of the ESEA. Thank you. 

Harry Gilbert: Thanks very much, Ted. And now, speaking 

for group three, Murray Rockowitz. 

Murray Rockowitz : The one unanimous thing about the 

group* s action was that I was to be its spokesman, for better or for 
worse, and the people who made the decision were Drs. Brown, Mitzel, 
Kalick, Lang, Lucey and Bray. I don*t think that Dr. Lucey has been 
introduced. Would he stand and take a bow? He 'sphere for the 
Council of Supervisory Associations of New York City. 

Our concerns were similar to those expressed earlier by. 

Dr. Schiff and by Dr. Lang and if I repeat some of them, I think. it's 
important to repeat them because of the fact that the emphases will 

emerge . 
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There was concern for the necessity for resources. Dr. 
Bray, the industry representative, started with hard cash as the 
basic to any research that could follow, and, armed with that caoh, 
he would look for a place to employ it. We all agreed that the 
climate where this research should take place would have to be a 
favorable climate, one not susceptible to pressures from various 
groups. Just where to find that, we leave to later deliberation. 

There was further agreement with respect to the kinds of 
research that should take place, and there we identified three dif- 
ferent kinds of research. Everyone agreed that the research that 
Dr. Flanagan outlined, basic longitudinal research, was essential, 
and that should go on regardless of any other consideration, and Dr. 
Kalick felt that the colleges could be counted upon in the actual 
effectuation of a good deal of the research program that would be 
developed. Dr. Mitzel stressed the two- to three-year intermediate 
type of research that would be based on process variables, perhaps, 
in observation of teacher behavior in the classroom as a starting 
point. He would stress that the New York City Board of Examiners, 
for example, move off in directions that would result from new con- 
ceptualizations of the examining process, all the while continuing 
with current procedures because of the necessities and exigencies of 
contemporary needs. 

There is a third type of research, and there’s a difference 
in the committee on this. The researchers feel that this third type 
of research is non-productive. The people more close to the firing 
line feel that it’s necessary, I refer to the ad hoc type of re- 
search which seeks to dissipate generalizations which are generated 
by emotions and pressures of temporary circumstances with a cold 
douche of fact. I think, too, that we have to get a climate where 
post- employment, the teaching staff is willing, with certain guaran- 
tees, to submit itself to research techniques, being reassured that 
the data that are being asked of them will in no way prejudice their 
status in the schools. 



i 




And the final point that was made by Dr. Brown is, I think 
this was unanimous too, that there are problems which are common to 
the largest districts in this country and this entire session will 
have been wasted if we just let it drop here. It is essential that 
we have a dialogue, a continuing dialogue, and that there be follow- 
up meetings of these groups to implement these suggestions, and I 
think Dr. Lang made the point earlier, step-by-step implementation 
over a period of time with specific goals to be set from meeting to 
meeting. 



Harry Gilbert: Thank you, Murray, 

the no committee, Dave Darland. 



And now, the friend of 
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David Darland: Thank you. As a group, we were y ery action- 

centered in our outlook. We felt that there were specific decisions 
that must be made, especially at crucial points in the process of se- 
lection of teachers . These points included selection itself, assign 
«£nt the earning of tenure, etc. We felt we knew a great deal more 
“out ££ to*^ develop programs related to these areas than we te^ to 
rkr»nr+-ire There was complete agreement that xiiiere had to 
fiSSTi tEaHhe characteristics and needs of any defined coMmnity 
woild be met This would include various interest groups, teachers 

associations , etc. 

Our group was also in agreement that the problems of selec- 
tion, and all its related problems, were so immense m “ irfeedT thwe 
nrvshibl V no city, no district or unit can solve them. Indeed, tnere 
considerable support for a national educational manpower study. 

Such a study might include utilization of personnel and giv ® 
priority to research, particularly the type that would P r °v 
predictive analyses.' Such a study would of cours^ have to incite 
the dvnamics of current society, recognizing that many of the tilings 
m havetoied in an attempt to solve the educational manpower 
shortage just haven* t worked. 

Our gr. ip underscored again the importance of twokindsof 
research, namely, applied and basic. There was considerable discus- 
sion on the importance of defining problems in an appropriati 8 
tert °i.e? e broad enough to really get at them effectively rather 
than' continuing to work on a piecemeal basis. In this regard, it is 
interesting to know that Louis Bright, the Associate Commissioner 
for Research of the United States Office of Education, has made the 
observation ^ 1 that, in the case of industry, after the basic research 
is done, it costs seven times as much to disseminate and innovate, 
onrt annlv the idea as it does to do the basic research in the first 
Dlace PP He^eels that we are falling down in education since we are 
not°paying enough attention to the application aspects of research. 
Possibly this somewhat corresponds to the thinking of our group. 

Thank you. 

Harry Gilbert: Thank you, Dave. I thank each of the Chair- 

men, Ai Schiff, Bob Perz, Bill Brown, and Dick Sanders for real y 
getting the groups to move. 

We have about five minutes and I have a few closing remarks 

consideration for a long time at the Board of Examiners. 

Some specifics. We would hope that each and evar yjiar- 
ticipant to the Invitational Conference would take the time tfien ^h 
returns home to write to the Conference Secretary, Mrs. Klein. We 
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suggest that you consider such things in the letter as, has this 
been a stimulus to any local action; secondly, has this been a 
stimulus to any action in organizations with which you are affili- 
ated; third, what specific suggestions do you have for follow-up; 
and fourth, do you have any specific suggestions for the composi- 
tion for the final report? 

This is by way of announcing that there will be a report. 
We have each of the papers, the formal papers that were presented 
or summarized. These will be published in the proceedings. We 
have the comments that everybody has made, and may I suggest that 
we will assume that we have your permission to include the comments 
that have been recorded, unless you specifically direct us other- 
wise. You may wish not to be reported or recorded. Yes, we will 
edit them rather carefully, and if there is any doubt, we will com- 
municate with you, but you’ll have to leave some discretion with 
the editorial task force. 

I want at this very last minute to repeat the thanks that 
Paul Denn and others have voiced. We thank Perry Kalick and Ger- 
hard Lang. I want to thank Paul Fitzgerald particularly, for the 
many hours that he gave to this work. I want to thank our gracious 
secretary, Mrs. Klein. I want to thank the Board of Examiners for 
their participation, and Jerry Eiseriberg, here, who has come up here 
given a couple of days, and has been working with back to the pro- 
ceedings, eager to get into the discussion, but he has stuck to his 
guns, or I should say, his tapes. Thank you very much, Jerry. 

And my final thanks, of course, go to the participants. 

You are all busy executives . That you have taken the time to pre- 
pare for this Conference and to participate in it is a tribute to 
you and to your recognition of the significance of the topic. On 
behalf of the Board of Examiners I thank you very much and hope you 
have a very pleasant trip home. 
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SECTION IV: REVIEW OF CONFERENCE 



,, . _ t times lengthy, comments on the variety 

The diverse, and at times leng y, g it di ffi CU lt 

of topics which occupied f ideas. This review consists 

to present their discussions, 2) a blueprint 

Synthesis of the Discussions 

Conference participants all agreed that teacher selection 

must be dealt with in over f ^ ^ eX ^ f 0 n!the!job trai^?« dur- 
teacher recruitment, teac ^® r p * prob i e ms inherent in teacher selec- 
ing the probationaryperi ^ philosop hical , psychological, politica , 
tion involve basic issues ox P“ one cannot grapple 

sociological, as well as economc d ^nsions^ On^ ^ with 

meaningfully with teacher se lection * tea cher selection," ra- 
the total gestalt, ® ^" h ^gedfrom the dark ages." 

^fesStsTf'the^o’ surveys^ 1 w£ich served as bases for this con- 
ference, reinforce this pessimistic observation. 

A number of agencies and grorps of peopleware, wittingly or 

unwittingly, partners an the teac er sel ^ institutions, 3) 

dates for teaching positions, 2) teacher “^[ ners)j U) boards of 
teacher selection agencies (e.g., who appoint teachers to 

education (including •»> supervisors, 

El^Se^epTt^ 

roles of s^ £m 

or hindering the teacher selection process. 



Oilbert, H. B.; Bogen, X.; : /school Systems 

Selection Polici es and Proc ur es In Lar^e PuDl . ^ ; - 

Education, E Department of Health, 

Education and Welfare, 1966. 63 p. 

J „ , . . p M Teacher Selection Policies and Procedures 
Lang, G. and Kalick, P. M. T» — - Metropolitan Scnool 

Si xty-two S c hool Syste ms Affiliated wicn one 1 1 

Council 



tKZ United States. New iui*w ***- 7171” oo « 
53ucaiion of iKT City of New York, 1966. 28 p. 
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Candidates for Teaching Positions 



The image of teaching as a career is still quite tarnished. 

Despite some adjustments in recent years, teacher salaries are still 
too low and are not attracting bright young people who are veering 
toward more lucrative fields. The real or imaginary image of the 
blackboard jungle is a greater deterrent to selecting teachers for 
the large urban schools than are possible inadequacies of the selec- 
tion instruments. Moreover, the selection process itself tends to be 
often viewed as a barrier. These and other factors resulted in the 
observation made by all representatives of the larger urban schools: 
we have a shortage of teachers, let alone "effective" teachers, and a 
sizeable number of .successful candidates (having passed the screening 
process) are lost to the more attractive suburban communities. The 
participants did not deal with the dynamics of vocational choice, 
i.e., why people select teaching. Teacher recruitment begins already 
in the elementary school. Teaching is the only profession which fur- * 

nishes positive or negative models to all people. The "good" and the j 

"bad" teachers who fill our classrooms are the unofficial recruiters 
and initial selectors. j 

Appointments to teaching positions in large cities must be 
made earlier than they are now made. May is the modal month for Sep- 
tember appointments. These should be made in March. 

Teacher Training Institutions 

Teacher selection is a developmental process with at least 
three stages: 1) selection of students for teacher preparatory pro- 
grams, 2) selection for the initial job, and 3) selection during the j 

probationary period culminating with the award of tenure. There must j 

be better screening at the entry level into the profession. Teacher ] 

training curricula must be modified better to prepare beginning j 

teachers for the realities of the classroom. Th3 recently announced 
programs by Fordham University and The City College of the City Uni- 
versity of New York as well as others throughout the nation, are 
steps in the right direction. These programs are emphasizing more 
experiences in the schools for students. Representatives from the 
colleges and the school systems issued a call for much closer co- 
operation and coordination of efforts, a continuous dialogue of these 
two partners. 

Teacher Selection Agencies 

The following questions were posed: 1) What is the role of 

the teacher selection agency? 2) What are the candidates' perceptions 
of the selection agency? 3) Who should be selected? U) What methods 
are and should be used? 5) How should we validate selection methods? 

* 
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1) Role of Teacher Selection Agency 



The teacher selection agency must consider that its selec- 
tions have to be made within a framework of reality, which was 
spelled out in great detail by Dr. Redfern. Several administrators 
from the larger school systems joined him in this overriding consid- 
erations current, and most likely future, teacher selection methods 
represent a compromise between what we are able to do, can do, and 
should be doing. 

The representatives from civil service voiced the unanimous 
opinion that the merit system must always be an important component 
of the selection process. Even in times of teacher shortages, there 
exist oversupplies of candidates for certain teaching areas and, 
usually, for supervisory positions. Lacking definitive selection 
criteria, at the very least, teacher selection agencies must devise a 
selection process which screens out the "poor risks,"' e.g., candi- 
dates who lack basic communication skills, who are severely physically 
or psychologically handicapped, and who might be prejudiced against 
the kinds of children whom they will be teaching. 

2) Candidates and the Teacher Selection Agency 

Intensive efforts must be exerted to "sell" the selection 
process to the candidates and to improve and modify (streamline) pro- 
cedures wherever possible. It is essential to discourage a seemingly 
widespread impression among candidates that they have to battle an 
impersonal system which seems to place too many roadblocks in the way 
of getting a teaching appointment. We must be cautious lest these at- 
titudes toward the selection agency and its process are carried over 
into teaching. For the candidate’s contact with the selection agency 
is his first contact with the "system" and it should, hopefully, be a 
mutually satisfactory one. 

3) Who is to be Selected 

The participants enumerated the usual array of characteris- 
tics presumed desirable for "effective" teachers. Three qualities 
were mentioned most frequently: commitment to teaching, ability to 

grow professionally, and flexibility. We should aim our efforts at 
the potential career teacher, not the temporary "drop-ins." We are 
selecting teachers for the 70' s and 80's. Teacher roles are most 
likely to change in direction and emphasis. To educate children who 
can cope with the demands and uncertainties of the future, we must 
select teachers who themselves have the requisite attitudes and skills 
to deal with the demands and opportunities which the future will 
bring. Selection of paraprofessional personnel will develop into a 
major task, it is believed. 
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h) Methods Which Are and Should Be Used 



Current methods, described in the two survey reports/ seem 
to screen out the "poor risks." There was a general consensus, how- 
ever, that our selection methods do not assess the primary qualities 
we are looking for in our candidates. The representatives from in- 
dustry cited techniques which school systems could experiment with. 

One of the most promising of these is the job simulation technique as 
exemplified by A. T. and T.'s assessment centers. Teacher assessment 
centers could be used as research criteria against which one could 
validate techniques and indexes used in the employment situation. Use 
of the video recorder as a means of training interviewers was another 
highly relevant suggestion offered. 

£) Validation of Teacher Selection Methods 

Agreement on valid criteria of effective selection methods 
and the development of adequate validating instruments has stymied 
researchers over a period of at least $0 years. This conference has 
not produced any quick or simple solutions. However, it did wrestle 
with this issue at practically every session. Three researchers out- 
lined specific paradigms for validating teacher selection methods. 

It was strongly suggested that the aptitude test model be abandoned 
for the achievement model, that we focus on content, not predictive 
validity, and that we use measures of classroom performance derived 
from systematic classroom observation as data sources. The problem 
of validating selection devices can and should be tackled at various 
levels of complexity. In many instances we must settle for practical, 
immediate criteria and use crude instruments. The real breakthrough 
in this area, however, can come only by a broad, not piecemeal, at- 
tack as outlined in the position papers of Drs. Ryans, Medley, Mitzel, 
and Flanagan. Such a frontal attack must be a cooperative venture 
joined by school systems, teacher training institutions, and led by 
the hard-core researchers: All of the experts agreed thao "over? 

teacher effectiveness" was a useless concept. Instead, specific ex- 
amples of teacher behavior in specific settings should be used as 
criteria and selection instruments devised to predict success m these 
limited and circumscribed areas. 



Boards of Education 

Two of the many tasks of boards of education were explored: 
1) to make teaching attractive, and 2) to provide a climate which is 
conducive to research on teacher selection problems as well as the 
means to conduct research. 



2 Ibid . 
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Efforts to evolve effective teacher selection methods and 
to maintain high selection standards are, to a large extent, vitiated 
by the persistent shortage of candidates to man the 
Wee cities. The union representatives and several school admi 
trators emphasized the weak holding power of our larger school sys- 
tems. Boards of education must concentrate on making ea ° ^ ^ . 

+ trorti ve Several avenues are open, such as offering signifies y 

h^er saarilsI IppoLting neophyte teachers to schools where there 

exists a maximization of probability of success or , ^’^ oes 

not appointing them to "difficult" schools which baffle the resources 
o7”hiehlv experienced teachers, and by enabling teachers to teach, 
fe g prLXg for the teacher 's physical safety, creating smaller 
ciasses^°employing teacher aides so that he can concentrate on 
instructional duties , etc . 

A number of paradigms for research on te “ h erselection 
were proposed. The boards of education must provide staffing and re 
sources to urriertake the needed research and to protect the re 
searchers from attacks by pressure groups which may wish to 
the thrust, the methodology used, and the reporting of data 
their own ends rather than those of the teaching profession and of 

society at large. 

School Administrators an d Supervisors 

School administrators and supervisors play crucial roles 
in the developmental teacher selection process . The Participants re- 
garded the probationary years as an intensiveon-the-jobtraining 
period as well as a golden opportunity to validate the initial screen 
ing efforts at the college and at the job entry levels. It was felt 
that teacher selection methods could be greatly improved by fo u ng 
Abetter methods of selecting school administrators «!>££££• 

Uncommitted and incompetent supervisors vitiate the best of recru 
t selection efforts. Inexperienced teachers who feel that 

they ^ floundering and are failing will not remain long in teaching. 
One^director of teacher training called for an intensive and =°">pre- 
hensiTO program for school administrators to take the place of the 
alito common hodgepodge of courses which prospective administrators 
take to fulfill certification requirements. 

Teachers^Unlons ■ Professional Orga nizations and the Educational 

The participants recognized that teachers unions , Profes- 
sional organizations, and members- of the "educational establishment" 

(e.g.. accrediting agencies and state departments .f^eir role S 
kev roles in teacher selection. It was suggested that their role i 
the '.arious decision-making processes be thoroughly studied. H 
this conference did not delve into this area. The efforts of two 
militant organizations of teachers, represented at this confers , 
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are well known with respect to upgrading teachers' salaries, reducing* 
class size, greater per-pupil expenditure, and greater teacher involve- 
ment in educational policy-making, just to cite a few illustrations. 



Community 



The conference touched upon but did not come to grips with 
knotty issues such as the role of laymen in teacher selection, citi- 
zens' expectations of teachers, and the use and the selection of 
paraprofessional teacher aides. These and many other facets were 
relegated to deliberations which should be held in the future. 



Blueprint for Action 



A number of concrete suggestions emerged from the con- 
ference which are recorded in Session 8s Summation of Conference, 
Additional recommendations follow: 



1) The problem of teacher selection must be attacked in a 
systematic manner, as spelled out in detail by Drs. Ryans, Mitzel, and 
Flanagan. 



2) Teacher selection methods should be validated. Appro- 
priate paradigms were offered by Dr. Ryans, Medley, Mitzel, Flanagan, 
and Gilbert. 



3) Research efforts should be launched on the selection of 
school administrators and supervisors. 



h) A number of recommendations were offered designed to 
cope with immediate problems (e.g., use of walk- in- tests y better train- 
ing of interviewers by use of the video recorder, job simulation, etc.), 



5) A working partnership should be formed among teacher 
training institutions, school systems and research workers drawn from 
diverse disciplines for the purpose of carrying out the major studies 
outlined in various position papers. 



6) Regional seminars should be held for leaders in educa- 
tional research, in allied disciplines, and school personnel adminis- 
trators to bridge the gap in their respective conceptualizations. 



7) A conference should be convened which deals with effec- 
tive utilization of educational manpower. Since there appear to 
exist a good deal of uncoordinated and wasted effort in the field of 
teacher selection and considerable personnel pirating, such a confer- 
ence seems very much needed. 



J 



* 



8) Annual meetings, like this conference, should be or- 
ganized, each meeting to have a specific program and course of action 
for the ensuing year to be assessed at the next meeting. 
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Evaluation of the Conference 

An evaluation must, of necessity at this time, be purely 
subjective. It is too early to see which seeds will come to fruition. 

All participants expressed a great deal of satisfaction 
with the conference. They felt it afforded a much-needed opportunity 
to exchange ideas. But, as must be expected, a nu^^ of negativ 
comments were also expressed. One participant felt that the college 
representatives and the school administrators were too defensive and 
the researchers too theoretical. Another person depiored the ap- 
parent greater tendency to analyze and agonize over existing problems 
than to seek action programs which he felt might lead to solutions. 

Time, of course, will provide the answer. In the meanwhile, 
it may be that the very many ideas - often conflicting - which W ®J® 
expressed at the conference will serve to stimulate readers who di 
not attend as well as those who did. 
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SECTION V: LIST OF PARTICIPANTS 



Baker, Paul - Manager, Social Science Research Division, Standard 

Oil Company of New Jersey 

Baldwin, H. W. - Supervisor, Certificated Selection Department, 

Los Angeles City School Districts 

Benson, Arthur L. - Senior Director, National Teacher Examinations, 

Educational Testing Service 

Berger, Bernard - Director of Training and Career Development Divi- 
sion, Department of Personnel, The City of New York 

Bransford, Thomas L. - Director of Examinations, New York State 

Civil Service Department 

Bray, Douglas W. - Director of Personnel Research, American Telephone 

and Telegraph Company 

Brown, William B. - Associate Superintendent of Schools, Los Angeles 

City Schools 

Brownell, Samuel N. - Professor of Urban Educational Administration, 

Graduate School, Yale University 

Browning, Rufus C. - Assistant Superintendent, Public Schools of the 

District of Columbia 

Cogen, Charles - President, American Federation of Teachers 

Darland, David D. - Editor, Journal of Teacher Education, National 

Education Association of the U. S. 

Denn, Paul - Vice Chairman, Board of Examiners, Board of Education 

of The City of New York 

Eisenberg, Jerrold - Acting Instructional Recordist, Bureau of Audio- 

Visual Instruction, Board of Education of The City 
of New York 

Entenmann, Thomas E. - Acting Director, Division of Examinations, 

The School District of Philadelphia 

Fitzgerald, Paul - Assistant Professor, Department of Education, 

St. John's University 

Flanagan, John C. - President, American Institutes for Research in 

the Behavioral Sciences 

Gilbert, Harry B. - Assistant Dean for Resident Instruction and Pro- 
fessor of Education, Pennsylvania State University 

Gold, Irving J. - Acting Examiner, Board of Examiners, Board of Edu- 
cation of The City of New York 

Gold, Milton - Director of Teacher Education, Hunter College of The 

City University of New York 

Greene, Jay E. - Examiner, Board of Examiners, Board of Education of 

The City of New York 

Jacobson, Raymond - Deputy Director, Bureau of Recruiting and Examin- 
ing, U. S. Civil Service Commission 

Kalick, Perry M. - Assistant Professor, Department of Education, 

Hunter College of The City University of New York 

Keating, Richard - Education Specialist, U. S. Office of Education 







Klein, Arthur - Examiner, Board of Examiners, Board of Education of 

The City of New York 

Kolodny, Jules - Secretary and Assistant to the President, United 

Federation of Teachers 

Lang, Gerhard - Associate Professor, Department of Education, Mont- 

clair State College 

Lang, Theodore H. - Deputy Superintendent of Schools, Board of 

Education of The City of New York 

Lierheimer, Alvin P. - Director, Division of Teacher Education and 

Certification, The University of the State of New 

York 

Lucey, Stuart - Assistant Superintendent of Schools, Board of Educa- 
tion of The City of New York 

Mackey, Charles - The State Education Department, The University of 

the State of New York 

Medley, Donald - Head, Teacher Behavior Research Group, Educational 

Testing Service 

Mitzel, Harold E. - Assistant Dean of Research, College of Education, 

and Professor of Psychology, Pennsylvania State 

University 

Perz, Robert E. - Associate Superintendent for Professional and 

Classified Personnel, Division of Personnel, 

Board of Education, Philadelphia 

Redfern, George - Associate Secretary, American Association of School 

Administrators 

Resnick, Lawrence — Chairman and Professor of Philosophy, State 

University of New York at Cortland 

Rivlin, Harry N» — Dean, School of Education, Fordham University 

Rockowitz, Murray - Examiner, Board of Examiners, Board of Education 

of The City of New York 

Ryans, David G. - Director, Education Research and Development Center, 

College of Education, University of Hawaii 

Sanders, Richard H. - Vice Chairman, Board of Examiners, Board of 

Education, Chicago 

Schiff, Albert - Director, Division of Personnel, Detroit Public Schools 

Steiner, Arch K. - Associate Research Coordinator, Research Branch, 

Division of Elementary and Secondary Research, 



U. S. Office of Education 

Streicher, Samuel - Examiner, Board of Examiners, Board of Education 

of The City of New York 

Unser, Gertrude - Examiner, Board of Examiners, Board of Education of 

The City of New York 

Walter, Ralph - Chairman, Department of Education, Montclair State 

College 

Williams, Fred H. - Assistant Superintendent, Board of Education of 

The City of New York 







